Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems

• Published in: IEEE transactions on parallel and distributed systems Vol. 32; no. 1; pp. 59 - 71
• Main Authors: Duan, Moming, Liu, Duo, Chen, Xianzhang, Liu, Renping, Tan, Yujuan, Liang, Liang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Balancing; Communications traffic; Data augmentation; Data models; Datasets; Distributed databases; distributed machine learning; Electronic devices; Federated learning; Machine learning; Mobile communication systems; Mobile handsets; Neural networks; Privacy; Rescheduling; Servers; Training
• ISSN: 1045-9219; 1558-2183
• DOI: 10.1109/TPDS.2020.3009406

Federated learning (FL) is a distributed deep learning method that enables multiple participants, such as mobile and IoT devices, to contribute a neural network while their private training data remains in local devices. This distributed approach is promising in the mobile systems where have a large corpus of decentralized data and require high privacy. However, unlike the common datasets, the data distribution of the mobile systems is imbalanced which will increase the bias of model. In this article, we demonstrate that the imbalanced distributed training data will cause an accuracy degradation of FL applications. To counter this problem, we build a self-balancing FL framework named Astraea, which alleviates the imbalances by 1) Z-score-based data augmentation, and 2) Mediator-based multi-client rescheduling. The proposed framework relieves global imbalance by adaptive data augmentation and downsampling, and for averaging the local imbalance, it creates the mediator to reschedule the training of clients based on Kullback-Leibler divergence (KLD) of their data distribution. Compared with FedAvg , the vanilla FL algorithm, Astraea shows +4.39 and +6.51 percent improvement of top-1 accuracy on the imbalanced EMNIST and imbalanced CINIC-10 datasets, respectively. Meanwhile, the communication traffic of Astraea is reduced by 75 percent compared to FedAvg .