Deep Covariance Alignment for Domain Adaptive Remote Sensing Image Segmentation

• Published in: IEEE transactions on geoscience and remote sensing Vol. 60; pp. 1 - 11
• Main Authors: Wu, Linshan, Lu, Ming, Fang, Leyuan
• Format: Journal Article
• Language: English
• Published: New York IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Alignment; Covariance; Deep covariance alignment (DCA); Dimensions; Divergence; Domains; Feature extraction; Generative adversarial networks; Image processing; Image segmentation; Regularization; Remote sensing; remote sensing image (RSI); semantic segmentation; Semantics; Spatial discrimination; Spatial resolution; Task analysis; Training; unsupervised domain adaptive (UDA)
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2022.3163278

Unsupervised domain adaptive (UDA) image segmentation has recently gained increasing attention, aiming to improve the generalization capability for transferring knowledge from the source domain to the target domain. However, in high spatial resolution remote sensing image (RSI), the same category from different domains (e.g., urban and rural) can appear to be totally different with extremely inconsistent distributions, which heavily limits the UDA accuracy. To address this problem, in this article, we propose a novel deep covariance alignment (DCA) model for UDA RSI segmentation. The DCA can explicitly align category features to learn shared domain-invariant discriminative feature representations, which enhance the ability of model generalization. Specifically, a category feature pooling (CFP) module is first used to extract category features by combining coarse outputs and deep features. Then, we leverage a novel covariance regularization (CR) to enforce the intracategory features to be closer and the intercategory features to be further separate. Compared with the existing category alignment methods, our CR aims to regularize the correlation between different dimensions of the features, and thus performs more robustly when dealing with divergent category features of imbalanced and inconsistent distributions. Finally, we propose a stagewise procedure to train the DCA to alleviate error accumulation. Experiments on both rural-to-urban and urban-to-rural scenarios of the LoveDA dataset demonstrate the superiority of our proposed DCA over other state-of-the-art UDA segmentation methods. Code is available at https://github.com/Luffy03/DCA .