DQC-ADMM: Decentralized Dynamic ADMM With Quantized and Censored Communications

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 8; pp. 3290 - 3304
• Main Authors: Liu, Yaohua, Wu, Gang, Tian, Zhi, Ling, Qing
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Alternating direction method of multipliers (ADMM); Communication; communication censoring; Communications; Convergence; Convex functions; decentralized network; dynamic optimization; Heuristic algorithms; Iterative methods; Linear programming; Mathematical analysis; Optimization; quantization; Quantization (signal); System dynamics; Time-varying systems; Tracking
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3051638

In distributed learning and optimization, a network of multiple computing units coordinates to solve a large-scale problem. This article focuses on dynamic optimization over a decentralized network. We develop a communication-efficient algorithm based on the alternating direction method of multipliers (ADMM) with quantized and censored communications, termed DQC-ADMM. At each time of the algorithm, the nodes collaborate to minimize the summation of their time-varying, local objective functions. Through local iterative computation and communication, DQC-ADMM is able to track the time-varying optimal solution. Different from traditional approaches requiring transmissions of the exact local iterates among the neighbors at every time, we propose to quantize the transmitted information, as well as adopt a communication-censoring strategy for the sake of reducing the communication cost in the optimization process. To be specific, a node transmits the quantized version of the local information to its neighbors, if and only if the value sufficiently deviates from the one previously transmitted. We theoretically justify that the proposed DQC-ADMM is capable of tracking the time-varying optimal solution, subject to a bounded error caused by the quantized and censored communications, as well as the system dynamics. Through numerical experiments, we evaluate the tracking performance and communication savings of the proposed DQC-ADMM.