Communication Efficient Asynchronous ADMM for General Form Consensus Optimization

• Published in: Ji suan ji ke xue Vol. 49; no. 11; pp. 309 - 315
• Main Authors: Wang, Dong-Xia, Lei, Yong-Mei, Zhang, Ze-Yu
• Format: Journal Article
• Language: Chinese
• Published: Chongqing Guojia Kexue Jishu Bu 01.11.2022; Editorial office of Computer Science
• Subjects: Algorithms; Big Data; Communication; distributed alternating direction method of multipliers|general form consensus optimization|sparse allreduce|hybrid programming model|logistic regression; Machine learning; Mathematical models; Multipliers; Optimization; Parameters
• ISSN: 1002-137X
• DOI: 10.11896/jsjkx.211200006

The distributed alternating direction method of multipliers（ADMM） is one of the most widely used methods for solving large-scale machine learning applications.However, most distributed ADMM algorithms are based on full model updates.With the increasing of system scale and data volume, the communication cost has become the bottleneck for the distributed ADMM when big data are involved.In order to reduce the communication cost in a distributed environment, a general form consensus asynchronous distributed alternating direction method of multipliers（GFC-ADADMM） is proposed in this paper.First, in the GFC-ADADMM,the associated model parameters rather than full model parameters are transmitted among nodes to reduce the transmission load, and the associated model parameters are filtered according to the characteristics of high-dimensional sparse data sets to further reduce the transmission load.Second, the GFC-ADMM is implemented by an asynchronous allreduce framework, which combines the advantage of the asynchrono