Perception and sensing for autonomous vehicles under adverse weather conditions: A survey

• Published in: ISPRS journal of photogrammetry and remote sensing Vol. 196; pp. 146 - 177
• Main Authors: Zhang, Yuxiao, Carballo, Alexander, Yang, Hanting, Takeda, Kazuya
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.02.2023
• Subjects: Adverse weather conditions; Autonomous driving; Deep learning; LiDAR; Perception and sensing; Sensor fusion; Deep learning; Sensor fusion; Perception and sensing; Autonomous driving; Adverse weather conditions; LiDAR
• ISSN: 0924-2716; 1872-8235
• DOI: 10.1016/j.isprsjprs.2022.12.021

Automated Driving Systems (ADS) open up a new domain for the automotive industry and offer new possibilities for future transportation with higher efficiency and comfortable experiences. However, perception and sensing for autonomous driving under adverse weather conditions have been the problem that keeps autonomous vehicles (AVs) from going to higher autonomy for a long time. This paper assesses the influences and challenges that weather brings to ADS sensors in a systematic way, and surveys the solutions against inclement weather conditions. State-of-the-art algorithms and deep learning methods on perception enhancement with regard to each kind of weather, weather status classification, and remote sensing are thoroughly reported. Sensor fusion solutions, weather conditions coverage in currently available datasets, simulators, and experimental facilities are categorized. Additionally, potential ADS sensor candidates and developing research directions such as V2X (Vehicle to Everything) technologies are discussed. By looking into all kinds of major weather problems, and reviewing both sensor and computer science solutions in recent years, this survey points out the main moving trends of adverse weather problems in perception and sensing, i.e., advanced sensor fusion and more sophisticated machine learning techniques; and also the limitations brought by emerging 1550 nm LiDARs. In general, this work contributes a holistic overview of the obstacles and directions of perception and sensing research development in terms of adverse weather conditions.