Learning Multi-Attention Context Graph for Group-Based Re-Identification

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 45; no. 6; pp. 7001 - 7018
• Main Authors: Yan, Yichao, Qin, Jie, Ni, Bingbing, Chen, Jiaxin, Liu, Li, Zhu, Fan, Zheng, Wei-Shi, Yang, Xiaokang, Shao, Ling
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Attention; Context; context learning; Context modeling; Datasets; Deep learning; Graph neural networks; Graphical representations; Group re-identification; Layout; Measurement; person re-identification; Robustness; Task analysis; Visualization
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2020.3032542

Learning to re-identify or retrieve a group of people across non-overlapped camera systems has important applications in video surveillance. However, most existing methods focus on (single) person re-identification (re-id), ignoring the fact that people often walk in groups in real scenarios. In this work, we take a step further and consider employing context information for identifying groups of people, i.e., group re-id. On the one hand, group re-id is more challenging than single person re-id, since it requires both a robust modeling of local individual person appearance (with different illumination conditions, pose/viewpoint variations, and occlusions), as well as full awareness of global group structures (with group layout and group member variations). On the other hand, we believe that person re-id can be greatly enhanced by incorporating additional visual context from neighboring group members, a task which we formulate as group-aware (single) person re-id. In this paper, we propose a novel unified framework based on graph neural networks to simultaneously address the above two group-based re-id tasks, i.e., group re-id and group-aware person re-id. Specifically, we construct a context graph with group members as its nodes to exploit dependencies among different people. A multi-level attention mechanism is developed to formulate both intra-group and inter-group context, with an additional self-attention module for robust graph-level representations by attentively aggregating node-level features. The proposed model can be directly generalized to tackle group-aware person re-id using node-level representations. Meanwhile, to facilitate the deployment of deep learning models on these tasks, we build a new group re-id dataset which contains more than <inline-formula><tex-math notation="LaTeX">3.8K</tex-math> <mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mi>K</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="qin-ieq1-3032542.gif"/> </inline-formula> images with <inline-formula><tex-math notation="LaTeX">1.5K</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mi>K</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="qin-ieq2-3032542.gif"/> </inline-formula> annotated groups, an order of magnitude larger than existing group re-id datasets. Extensive experiments on the novel dataset as well as three existing datasets clearly demonstrate the effectiveness of the proposed framework for both group-based re-id tasks.