Direction-Coded Temporal U-Shape Module for Multiframe Infrared Small Target Detection

• Published in: IEEE transaction on neural networks and learning systems Vol. PP; pp. 1 - 14
• Main Authors: Li, Ruojing, An, Wei, Xiao, Chao, Li, Boyang, Wang, Yingqian, Li, Miao, Guo, Yulan
• Format: Journal Article
• Language: English
• Published: United States IEEE 17.11.2023
• Subjects: Annotations; Clutter; Convolution; Direction coding; Feature extraction; infrared small target (IRST) detection; Learning systems; Object detection; point-level supervision; spatial–temporal fusion; Trajectory
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2023.3331004

Infrared small target (IRST) detection aims at separating targets from cluttered background. Although many deep learning-based single-frame IRST (SIRST) detection methods have achieved promising detection performance, they cannot deal with extremely dim targets while suppressing the clutters since the targets are spatially indistinctive. Multiframe IRST (MIRST) detection can well handle this problem by fusing the temporal information of moving targets. However, the extraction of motion information is challenging since general convolution is insensitive to motion direction. In this article, we propose a simple yet effective direction-coded temporal U-shape module (DTUM) for MIRST detection. Specifically, we build a motion-to-data mapping to distinguish the motion of targets and clutters by indexing different directions. Based on the motion-to-data mapping, we further design a direction-coded convolution block (DCCB) to encode the motion direction into features and extract the motion information of targets. Our DTUM can be equipped with most single-frame networks to achieve MIRST detection. Moreover, in view of the lack of MIRST datasets, including dim targets, we build a multiframe infrared small and dim target dataset (namely, NUDT-MIRSDT) and propose several evaluation metrics. The experimental results on the NUDT-MIRSDT dataset demonstrate the effectiveness of our method. Our method achieves the state-of-the-art performance in detecting infrared small and dim targets and suppressing false alarms. Our codes will be available at https://github.com/TinaLRJ/Multi-frame-infrared-small-target-detection-DTUM.