HiFlash: Communication-Efficient Hierarchical Federated Learning With Adaptive Staleness Control and Heterogeneity-Aware Client-Edge Association

• Published in: IEEE transactions on parallel and distributed systems Vol. 34; no. 5; pp. 1560 - 1579
• Main Authors: Wu, Qiong, Chen, Xu, Ouyang, Tao, Zhou, Zhi, Zhang, Xiaoxi, Yang, Shusen, Zhang, Junshan
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.05.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Adaptive control; Agglomeration; Client-edge association; Clients; Cloud computing; Communication; Communications systems; Computational modeling; Controllability; Convergence; Data models; Deep learning; Edge computing; Federated learning; Heterogeneity; hierarchical mechanism; Local area networks; Mobile computing; Model accuracy; Servers; staleness control; Staling; Wide area networks
• ISSN: 1045-9219; 1558-2183
• DOI: 10.1109/TPDS.2023.3238049

Federated learning (FL) is a promising paradigm that enables collaboratively learning a shared model across massive clients while keeping the training data locally. However, for many existing FL systems, clients need to frequently exchange model parameters of large data size with the remote cloud server directly via wide-area networks (WAN), leading to significant communication overhead and long transmission time. To mitigate the communication bottleneck, we resort to the hierarchical federated learning paradigm of HiFL, which reaps the benefits of mobile edge computing and combines synchronous client-edge model aggregation and asynchronous edge-cloud model aggregation together to greatly reduce the traffic volumes of WAN transmissions. Specifically, we first analyze the convergence bound of HiFL theoretically and identify the key controllable factors for model performance improvement. We then advocate an enhanced design of HiFlash by innovatively integrating deep reinforcement learning based adaptive staleness control and heterogeneity-aware client-edge association strategy to boost the system efficiency and mitigate the staleness effect without compromising model accuracy. Extensive experiments corroborate the superior performance of HiFlash in model accuracy, communication reduction, and system efficiency.