Spectrum-irrelevant fine-grained representation for visible–infrared person re-identification

• Published in: Computer vision and image understanding Vol. 232; p. 103703
• Main Authors: Gong, Jiahao, Zhao, Sanyuan, Lam, Kin-Man, Gao, Xin, Shen, Jianbing
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.07.2023
• Subjects: Visible–infrared person re-identification; 65D05; 65D17; Visible–infrared person re-identification; 41A05; 41A10
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2023.103703

Visible–infrared person re-identification (VI-ReID) is an important and practical task for full-time intelligent surveillance systems. Compared to visible person re-identification, it is more challenging due to the large cross-modal discrepancy. Existing VI-ReID methods suffer from heterogeneous structures and the different spectra of visible and infrared images. In this work, we propose the Spectrum-Insensitive Data Augmentation (SIDA) strategy, which effectively alleviates the disturbance in the visible and infrared spectra and forces the network to learn spectrum-irrelevant features. The network also compares samples with both global and local features. We devise a Feature Relation Reasoning (FRR) module to learn discriminative fine-grained representations according to the graph reasoning principle. Compared to the most commonly used uniform partition, our FRR better adopts to the case of VI-ReID, in which human bodies are difficult to align. Furthermore, we design the dual center loss for learning the global feature in order to maintain the intra-modality relations, while learning the cross-modal similarities. Our method achieves better convergence in training. Extensive experiments demonstrate that our method achieves state-of-the-art performance on two visible–infrared cross-modal Re-ID datasets. •Analyzing the cross-modality discrepancy and studying the data augmentation on spectra information, we propose a Spectrum-Insensitive Data Augmentation (SIDA) strategy.•We develop a Feature Relation Reasoning (FRR) module based on the graph reasoning principle, for extraction and alignment of the fine-grained representation. Through further transferring information among cross-modality samples on the part-level, FRR learns discriminative feature representations.•We present an effective solution for VI-ReID. The experiments demonstrate that our method achieves the state-of-the-art performance on two popular benchmarks of VI-ReID datasets.