Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients

• Published in: Ji suan ji ke xue Vol. 49; no. 9; pp. 183 - 193
• Main Authors: Lu, Chen-yang, Deng, Su, Ma, Wu-bin, Wu, Ya-hui, Zhou, Hao-hao
• Format: Journal Article
• Language: Chinese
• Published: Chongqing Guojia Kexue Jishu Bu 01.09.2022; Editorial office of Computer Science
• Subjects: Algorithms; Clients; Clustering; Convergence; Decay rate; Federated learning; federated learning|privacy protection|clustering|stratified sampling|distributed optimization|convergence analysis; Heterogeneity; Machine learning; Optimization; Sampling
• DOI: 10.11896/jsjkx.220500263
• ISSN: 1002-137X

Federated learning（FL） is a new distributed learning framework for privacy protection, which is different from traditional distributed machine learning: 1)differences in communication, computing, and storage performance among devices（device heterogeneity）,2)differences in data distribution and data volume（data heterogeneity）,and 3)high communication consumption.Under heterogeneous conditions, the data distribution of clients varies greatly, which leads to the decrease of model convergence speed.Especially in the case of highly heterogeneous condition, the traditional FL algorithm cannot converge and the training loss curve will fluctuate greatly with the increase of local iterations.In this work, a FL algorithm based on stratified sampling optimization（FedSSO） is proposed.In FedSSO,a density-based clustering method is used to divide the overall client into different clusters.Then, some available clients are proportionally extracted from different clusters to participate in training.Therefore, various data are