Illumination Distillation Framework for Nighttime Person Re-Identification and a New Benchmark

• Published in: IEEE transactions on multimedia Vol. 26; pp. 1 - 14
• Main Authors: Lu, Andong, Zhang, Zhang, Huang, Yan, Zhang, Yifan, Li, Chenglong, Tang, Jin, Wang, Liang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Cameras; Datasets; Distillation; Feature extraction; Illumination; Image enhancement; Learning; Lighting; Modules; Night; Robustness; Task analysis; Visual tasks; Visualization
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2023.3266066

Nighttime person Re-ID (person re-identification in the nighttime) is a very important and challenging task for visual surveillance but it has not been thoroughly investigated. Under the low illumination condition, the performance of person Re-ID methods usually sharply deteriorates. To address the low illumination challenge in nighttime person Re-ID, this paper proposes an Illumination Distillation Framework (IDF), which utilizes illumination enhancement and illumination distillation schemes to promote the learning of Re-ID models. Specifically, IDF consists of a master branch, an illumination enhancement branch, and an illumination distillation module. The master branch is used to extract the features from a nighttime image. The illumination enhancement branch first estimates an enhanced image from the nighttime image using a nonlinear curve mapping method and then extracts the enhanced features. However, nighttime and enhanced features usually contain data noise due to unstable lighting conditions and enhancement failures. To fully exploit the complementary benefits of nighttime and enhanced features while suppressing data noise, we propose an illumination distillation module. In particular, the illumination distillation module fuses the features from two branches through a bottleneck fusion model and then uses the fused features to guide the learning of both branches in a distillation manner. In addition, we build a real-world nighttime person Re-ID dataset, named Night600 , which contains 600 identities captured from different viewpoints and nighttime illumination conditions under complex outdoor environments. Experimental results demonstrate that our IDF can achieve state-of-the-art performance on two nighttime person Re-ID datasets ( i.e. , Night600 and Knight ). We will release our code and dataset at https://github.com/Alexadlu/IDF .