Error Compensated Loopless SVRG, Quartz, and SDCA for Distributed Optimization

• Main Authors: Qian, Xun, Dong, Hanze, Richtárik, Peter, Zhang, Tong
• Format: Journal Article
• Language: English
• Published: 21.09.2021
• Subjects: Mathematics - Optimization and Control

The communication of gradients is a key bottleneck in distributed training of large scale machine learning models. In order to reduce the communication cost, gradient compression (e.g., sparsification and quantization) and error compensation techniques are often used. In this paper, we propose and s...