Asynchronous distributed alternating direction method of multipliers: Algorithm and convergence analysis

• Published in: Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) pp. 4781 - 4785
• Main Authors: Chang, Tsung-Hui, Hong, Mingyi, Liao, Wei-Cheng, Wang, Xiangfeng
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.03.2016
• Subjects: Algorithm design and analysis; Algorithms; alternating direction method of multipliers; asynchronous algorithm; Conferences; Convergence; Delay; Delays; Distributed optimization; Electronic mail; Mathematical models; Multipliers; Networks; Optimization; Protocols
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP.2016.7472585

Alternating direction method of multipliers (ADMM) has been recognized as an efficient approach for solving many large-scale learning problems over a computer cluster. However, traditional synchronized computation does not scale well with the problem size, as the speed of the algorithm is limited by the slowest workers. In this paper, we propose an asynchronous distributed ADMM (AD- ADMM) which can effectively improve the time efficiency of distributed optimization. Our main interest lies in characterizing the convergence conditions of the AD-ADMM, under the popular partially asynchronous model which is defined based on a maximum tolerable delay in the network. Specifically, by considering general and possibly non-convex cost functions, we show that the AD-ADMM converges to the set of Karush-Kuhn-Tucker (KKT) points as long as the algorithm parameters are chosen appropriately according to the network delay. We also show that the asynchrony of ADMM has to be handled with care, as a slightly different implementation can significantly jeopardize the algorithm convergence.