Graph Sampling-Based Multi-Stream Enhancement Network for Visible-Infrared Person Re-Identification

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 18; p. 7948
• Main Authors: Jiang, Jinhua, Xiao, Junjie, Wang, Renlin, Li, Tiansong, Zhang, Wenfeng, Ran, Ruisheng, Xiang, Sen
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.09.2023; MDPI
• Subjects: Adaptability; Contour Expansion Module; Cross-modality Graph Sampler; Methods; modality discrepancy; Multi-Modal Data; Neural networks; Semantics; Surveillance; VI Re-ID
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23187948

With the increasing demand for person re-identification (Re-ID) tasks, the need for all-day retrieval has become an inevitable trend. Nevertheless, single-modal Re-ID is no longer sufficient to meet this requirement, making Multi-Modal Data crucial in Re-ID. Consequently, a Visible-Infrared Person Re-Identification (VI Re-ID) task is proposed, which aims to match pairs of person images from the visible and infrared modalities. The significant modality discrepancy between the modalities poses a major challenge. Existing VI Re-ID methods focus on cross-modal feature learning and modal transformation to alleviate the discrepancy but overlook the impact of person contour information. Contours exhibit modality invariance, which is vital for learning effective identity representations and cross-modal matching. In addition, due to the low intra-modal diversity in the visible modality, it is difficult to distinguish the boundaries between some hard samples. To address these issues, we propose the Graph Sampling-based Multi-stream Enhancement Network (GSMEN). Firstly, the Contour Expansion Module (CEM) incorporates the contour information of a person into the original samples, further reducing the modality discrepancy and leading to improved matching stability between image pairs of different modalities. Additionally, to better distinguish cross-modal hard sample pairs during the training process, an innovative Cross-modality Graph Sampler (CGS) is designed for sample selection before training. The CGS calculates the feature distance between samples from different modalities and groups similar samples into the same batch during the training process, effectively exploring the boundary relationships between hard classes in the cross-modal setting. Some experiments conducted on the SYSU-MM01 and RegDB datasets demonstrate the superiority of our proposed method. Specifically, in the VIS→IR task, the experimental results on the RegDB dataset achieve 93.69% for Rank-1 and 92.56% for mAP.