Multi-Spectral Band Selection and Spatial Explanations Using XAI Algorithms in Remote Sensing Applications

• Published in: IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium pp. 6093 - 6396
• Main Authors: Temenos, Anastasios, Temenos, Nikos, Kaselimi, Maria, Doulamis, Anastasios, Doulamis, Nikolaos
• Format: Conference Proceeding
• Language: English
• Published: IEEE 16.07.2023
• Subjects: Correlation; Deep learning; EuroSAT; Heating systems; Land Cover; Land Use; Remote Sensing; Rivers; Satellite images; Sensors; Sentinel-2; SHAP; Transformers; ViT; XAI
• DOI: 10.1109/IGARSS52108.2023.10282565

This work proposes an interpretable Deep Learning framework utilizing Vision Transformers (ViT) for the classification of remote sensing images into land use and land cover (LULC) classes. It uses the Shapley Additive Explanations (SHAP) values to achieve two-stage explanations: 1) bandwise feature importance per class, showing which band assists the prediction of each class and 2) spatial-wise feature understanding, explaining which embedded patches per band affected the network's performance. Experimental results on the EuroSAT dataset demonstrate the ViT's accurate classification with an overall accuracy 96.86 %, offering improved results when compared to popular CNN models. Heatmaps in each one of the dataset's existing classes highlight the effectiveness of the proposed framework in the band explanation and the feature importance.