Explaining the Effects of Clouds on Remote Sensing Scene Classification

• Published in: IEEE journal of selected topics in applied earth observations and remote sensing Vol. 15; pp. 9976 - 9986
• Main Authors: Gawlikowski, Jakob, Ebel, Patrick, Schmitt, Michael, Zhu, Xiao Xiang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Atmospheric disturbances; Atmospheric models; Classification; Cloud computing; Clouds; deep learning; Earth; explainability; Forestry; Haze; Imagery; Machine learning; Observational learning; Remote sensing; robustness; Task analysis; Wetlands
• ISSN: 1939-1404; 2151-1535
• DOI: 10.1109/JSTARS.2022.3221788

Most of Earth is covered by haze or clouds, impeding the constant monitoring of our planet. Preceding works have documented the detrimental effects of cloud coverage on remote sensing applications and proposed ways to approach this issue. However, up to now, little effort has been spent on understanding how exactly atmospheric disturbances impede the application of modern machine learning methods to Earth observation data. Specifically, we consider the effects of haze and cloud coverage on a scene classification task. We provide a thorough investigation of how classifiers trained on cloud-free data fail once they encounter noisy imagery-a common scenario encountered when deploying pretrained models for remote sensing to real use cases. We show how and why remote sensing scene classification suffers from cloud coverage. Based on a multistage analysis, including explainability approaches applied to the predictions, we work out four different types of effects that clouds have on scene prediction. The contribution of our work is to deepen the understanding of the effects of clouds on common remote sensing applications and consequently guide the development of more robust methods.