Who Cares about the Weather? Inferring Weather Conditions for Weather-Aware Object Detection in Thermal Images

• Published in: Applied sciences Vol. 13; no. 18; p. 10295
• Main Authors: Johansen, Anders Skaarup, Nasrollahi, Kamal, Escalera, Sergio, Moeslund, Thomas B.
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.09.2023
• Subjects: Cameras; Classification; concept drift; conditioning; Datasets; Heat detection; Humidity; Infrared radiation; object detection; Optimization; Sensors; thermal; Vision systems; weather recognition; Germany
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app131810295

Deployments of real-world object detection systems often experience a degradation in performance over time due to concept drift. Systems that leverage thermal cameras are especially susceptible because the respective thermal signatures of objects and their surroundings are highly sensitive to environmental changes. In this study, two types of weather-aware latent conditioning methods are investigated. The proposed method aims to guide two object detectors, (YOLOv5 and Deformable DETR) to become weather-aware. This is achieved by leveraging an auxiliary branch that predicts weather-related information while conditioning intermediate layers of the object detector. While the conditioning methods proposed do not directly improve the accuracy of baseline detectors, it can be observed that conditioned networks manage to extract a weather-related signal from the thermal images, thus resulting in a decreased miss rate at the cost of increased false positives. The extracted signal appears noisy and is thus challenging to regress accurately. This is most likely a result of the qualitative nature of the thermal sensor; thus, further work is needed to identify an ideal method for optimizing the conditioning branch, as well as to further improve the accuracy of the system.