Tracker Meets Night: A Transformer Enhancer for UAV Tracking

• Published in: IEEE robotics and automation letters Vol. 7; no. 2; pp. 3866 - 3873
• Main Authors: Ye, Junjie, Fu, Changhong, Cao, Ziang, An, Shan, Zheng, Guangze, Li, Bowen
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aerial systems: applications; Autonomous aerial vehicles; Benchmarks; data sets for robotic vision; deep learning for visual perception; Feature extraction; Image enhancement; Lighting; low-light enhancement; Nighttime construction; nighttime tracking; Optical tracking; spatial-channel transformer; Target tracking; Task analysis; Transformers; Unmanned aerial vehicles; Visualization
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2022.3146911

Most previous progress in object tracking is realized in daytime scenes with favorable illumination. State-of-the-arts can hardly carry on their superiority at night so far, thereby considerably blocking the broadening of visual tracking-related unmanned aerial vehicle (UAV) applications. To realize reliable UAV tracking at night, a spatial-channel Transformer-based low-light enhancer (namely SCT), which is trained in a novel task-inspired manner, is proposed and plugged prior to tracking approaches. To achieve semantic-level low-light enhancement targeting the high-level task, the novel spatial-channel attention module is proposed to model global information while preserving local context. In the enhancement process, SCT denoises and illuminates nighttime images simultaneously through a robust non-linear curve projection. Moreover, to provide a comprehensive evaluation, we construct a challenging nighttime tracking benchmark, namely DarkTrack2021, which contains 110 challenging sequences with over 100 K frames in total. Evaluations on both the public UAVDark135 benchmark and the newly constructed DarkTrack2021 benchmark show that the task-inspired design enables SCT with significant performance gains for nighttime UAV tracking compared with other top-ranked low-light enhancers. Real-world tests on a typical UAV platform further verify the practicability of the proposed approach. The DarkTrack2021 benchmark and the code of the proposed approach are publicly available at https://github.com/vision4robotics/SCT .