A communication-efficient distributed deep learning remote sensing image change detection framework

• Published in: International journal of applied earth observation and geoinformation Vol. 129; p. 103840
• Main Authors: Cheng, Hongquan, Zheng, Jie, Wu, Huayi, Qi, Kunlun, He, Lihua
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.05.2024; Elsevier
• Subjects: Change detection; Communication compression; Distributed deep learning; Parallel computing; Staleness compensation; Staleness compensation; Parallel computing; Change detection; Communication compression; Distributed deep learning
• ISSN: 1569-8432; 1872-826X
• DOI: 10.1016/j.jag.2024.103840

•A distributed deep learning change detection model training framework for network-constrained systems is proposed.•Gradient compression approaches are introduced to mitigate the restriction of the network on distributed change detection modeling.•A momentum compensation mechanism for distributed staleness updating is constructed in conjunction with theoretical analysis.•Experimental tests address the impacts of gradient compression methods, compression rates, and the compensation mechanism on distributed change detection. With the introduction of deep learning methods, the computation required for remote sensing change detection has significantly increased, and distributed computing is applied to remote sensing change detection to improve computational efficiency. However, due to the large size of deep learning models, the time-consuming gradient transfer during distributed model training weakens the acceleration effectiveness in change detection. Data communication and updates can be the bottlenecks in distributed change detection systems with limited network resources. To address the interrelated problems, we propose a communication-efficient distributed deep learning remote sensing change detection framework (CEDD-CD) based on the synchronous update architecture. The CEDD-CD integrates change detection with communication-efficient distributed gradient compression approaches, which can efficiently reduce the data volume to be transferred. In addition, for the implicit effect caused by the delay of compressed gradient update, a momentum compensation mechanism under theoretical analysis was constructed to reduce the time consumption required for model convergence and strengthen the stability of distributed training. We also designed a unified distributed change detection system architecture to reduce the complexity of distributed modeling. Experiments were conducted on three datasets; the qualitative and quantitative results demonstrate that the CEDD-CD was effective for massive remote sensing image change detection.