Unsupervised Nighttime Object Tracking Based on Transformer and Domain Adaptation Fusion Network

• Published in: IEEE access Vol. 12; pp. 130896 - 130913
• Main Authors: Wei, Haoran, Fu, Yanyun, Wang, Deyong, Guo, Rui, Zhao, Xueyi, Fang, Jian
• Format: Journal Article
• Language: English
• Published: IEEE 2024
• Subjects: Adaptation models; Feature extraction; feature fusion; Fuses; Siamese network; Single object tracking; Target tracking; Training; transformer; Transformers; UAV; Vectors
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3378117

In view of the complexity, uncertainty, and low visibility of the night-time environment. In this study, we designed an unsupervised domain adaptation framework, TransffCAR, using a Siamese network of the source (daytime) and target (nighttime) domains. Training patches are generated by finding targets from a large number of unlabelled frame images. This avoids heavy manual labelling, improves the performance of the network, and allows the tracker to adapt to different night scenes. We also developed a transformer connection layer, which can help the network align cross-domain features. Secondly, we propose an efficient and lightweight attention block in the Transformer discriminator to increase frames per second while keeping the metrics as constant as possible, while the domain adaptive principle is used in the discriminator to form adversarial learning to reduce the domain differences in order to correctly distinguish the source domain (daytime) and the target domain (nighttime). Finally, we propose an adaptive hierarchical feature fusion module that efficiently fuses adjacent three-level feature information to facilitate the generation of unambiguous predictive tracking maps for subsequent input to the area regression network. We chose to test it on the NAT2021 public dataset and the UAVDark70 public dataset and tested three evaluation metrics that outperformed other state-of-the-art tracking methods. And our proposed tracker is seen to be very competitive in terms of correctness improvement of 4.93%, 8.26%, 9.42% and 8.49% in the NAT2021-test test set compared to SOTA tracking such as UDAT-CAR, UDAT-BAN, SparseTT and MixFormer.In addition, we introduced a new dataset called DarkBorder to validate the generalizability of our designed method when dealing with the task of UAV nighttime aerial video tracking.