Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation

• Published in: IEEE/ACM transactions on networking Vol. 29; no. 1; pp. 398 - 409
• Main Authors: Dinh, Canh T., Tran, Nguyen H., Nguyen, Minh N. H., Hong, Choong Seon, Bao, Wei, Zomaya, Albert Y., Gramoli, Vincent
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computational modeling; Convergence; Data models; Distributed machine learning; Energy consumption; Federated learning; Heterogeneity; Machine learning; Optimization; optimization decomposition; Power consumption; Resource allocation; Resource management; Tradeoffs; Training; Wireless communication; Wireless networks
• ISSN: 1063-6692; 1558-2566
• DOI: 10.1109/TNET.2020.3035770

There is an increasing interest in a fast-growing machine learning technique called Federated Learning (FL), in which the model training is distributed over mobile user equipment (UEs), exploiting UEs' local computation and training data. Despite its advantages such as preserving data privacy, FL still has challenges of heterogeneity across UEs' data and physical resources. To address these challenges, we first propose FEDL , a FL algorithm which can handle heterogeneous UE data without further assumptions except strongly convex and smooth loss functions. We provide a convergence rate characterizing the trade-off between local computation rounds of each UE to update its local model and global communication rounds to update the FL global model. We then employ FEDL in wireless networks as a resource allocation optimization problem that captures the trade-off between FEDL convergence wall clock time and energy consumption of UEs with heterogeneous computing and power resources. Even though the wireless resource allocation problem of FEDL is non-convex, we exploit this problem's structure to decompose it into three sub-problems and analyze their closed-form solutions as well as insights into problem design. Finally, we empirically evaluate the convergence of FEDL with PyTorch experiments, and provide extensive numerical results for the wireless resource allocation sub-problems. Experimental results show that FEDL outperforms the vanilla FedAvg algorithm in terms of convergence rate and test accuracy in various settings.