Region Selective Fusion Network for Robust RGB-T Tracking

• Published in: IEEE signal processing letters Vol. 30; pp. 1357 - 1361
• Main Authors: Yu, Zhencheng, Fan, Huijie, Wang, Qiang, Li, Ziwan, Tang, Yandong
• Format: Journal Article
• Language: English
• Published: New York IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ad hoc networks; Deep visual tracking; Feature extraction; Head; Infrared imagery; Infrared tracking; Mobile computing; neural networks; Optical tracking; Redundancy; Reliability; Robustness; Target tracking; visible-infrared fusion; vision transformer; Visualization
• ISSN: 1070-9908; 1558-2361
• DOI: 10.1109/LSP.2023.3316021

RGB-T tracking utilizes thermal infrared images as a complement to visible light images in order to perform more robust visual tracking in various scenarios. However, the highly aligned RGB-T image pairs introduces redundant information, the modal quality fluctuation during tracking also brings unreliable information. Existing RGB-T trackers usually use channel-wise multi-modal feature fusion in which the low-quality features degrades the fused features and causes trackers to drift. In this work, we propose a region selective fusion network that first evaluates each image region by cross-modal and cross-region modeling, then removes low-quality redundant region features to alleviate the negative effects caused by unreliable information in multi-modal fusion. Besides, the region removal scheme brings a efficiency boost as redundant features are removed progressively, this enables the tracker to run at a high tracking speed. Extensive experiments show that the proposed tracker achieves competitive performance with a real-time tracking speed on multiple RGB-T tracking benchmarks including LasHeR, RGBT234 and GTOT.