An Efficiency-Boosting Client Selection Scheme for Federated Learning With Fairness Guarantee

• Published in: IEEE transactions on parallel and distributed systems Vol. 32; no. 7; pp. 1552 - 1564
• Main Authors: Huang, Tiansheng, Lin, Weiwei, Wu, Wentai, He, Ligang, Li, Keqin, Zomaya, Albert Y.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.07.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Client selection; Clients; Collaboration; Computer networks; Computer science; contextual combinatorial multi-arm bandit; Data models; Distributed processing; fairness scheduling; Federated learning; lyapunov optimization; Mathematical models; Optimization; Privacy; Problem solving; Servers; Training
• DOI: 10.1109/TPDS.2020.3040887
• ISSN: 1045-9219

The issue of potential privacy leakage during centralized AI's model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally, without the necessity to upload their personal sensitive data. In FL, the number of clients could be sufficiently large, but the bandwidth available for model distribution and re-upload is quite limited, making it sensible to only involve part of the volunteers to participate in the training process. The client selection policy is critical to an FL process in terms of training efficiency, the final model's quality as well as fairness. In this article, we will model the fairness guaranteed client selection as a Lyapunov optimization problem and then a <inline-formula><tex-math notation="LaTeX">\mathbf {C^2MAB}</tex-math> <mml:math><mml:mrow><mml:msup><mml:mi mathvariant="bold">C</mml:mi><mml:mn mathvariant="bold">2</mml:mn></mml:msup><mml:mi mathvariant="bold">MAB</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="lin-ieq1-3040887.gif"/> </inline-formula>-based method is proposed for estimation of the model exchange time between each client and the server, based on which we design a fairness guaranteed algorithm termed RBCS-F for problem-solving. The regret of RBCS-F is strictly bounded by a finite constant, justifying its theoretical feasibility. Barring the theoretical results, more empirical data can be derived from our real training experiments on public datasets.