Multilevel Knowledge Transmission for Object Detection in Rainy Night Weather Conditions

• Published in: IEEE transactions on industrial informatics Vol. 20; no. 9; pp. 11224 - 11232
• Main Authors: Le, Trung-Hieu, Huang, Shih-Chia, Hoang, Quoc-Viet
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.09.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Convolutional neural network (CNN); Datasets; Detectors; Feature extraction; Loss measurement; low illumination; Meteorology; Night; object detection (OD); Object recognition; Rain; rain removal (RR); rainy nighttime (RNT); Task analysis; Training; Weather
• ISSN: 1551-3203; 1941-0050
• DOI: 10.1109/TII.2024.3396552

In recent years, deep convolutional neural networks (CNNs) have been widely applied and have gained considerable success in object detection (OD). However, most of the CNN-based object detectors have been developed to operate under favorable weather conditions, limiting their ability to accurately detect objects in rainy nighttime (RNT) scenes, thereby resulting in low performance. In this work, we introduce a multilevel knowledge transmission network (MKT-Net) to overcome the challenges of detecting objects with the interference of rain and night. Our proposed model accomplishes this objective by collaborating OD with rain removal (RR) and low-illumination enhancement (LE) tasks. Specifically, the MKT-Net is composed of three main subnetworks that share some shallow layers with each other: an OD subnetwork for performing object classification and localization, an RR subnetwork, and an LE subnetwork for generating clear features. To aggregate and transmit multiscale features generated by the RR and LE subnetworks to the OD subnetwork for boosting detection accuracy, we introduce two feature transmission modules with identical architectures. Extensive evaluation on various datasets has demonstrated the effectiveness of our proposed model, which outperformed competing methods by up to 25.43% and 15.26% in mean average precision on a collected RNT dataset and the published rain in driving dataset, respectively, while maintaining high detection speed.