FedLC: Optimizing Federated Learning in Non-IID Data via Label-wise Clustering

• Published in: IEEE access Vol. 11; p. 1
• Main Authors: Lee, Hunmin, Seo, Daehee
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Agglomeration; Algorithms; Clustering; Clustering algorithms; Computer architecture; Convergence; Convergence Optimization in Federated Learning; Datasets; Federated learning; Federated Learning in Heterogenous and Non-IID Dataset; Geographical distribution; Label-wise Clustering; Optimization; Training
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2023.3271517

As contemporary systems are being operated in dynamic situations alternating into decentralized and distributed environments from conventional centralized frameworks, Federated Learning (FL) has been gaining attention for an effective architecture when aggregating the information and correspondingly trained AI model of geographically distributed datasets. However, the convergence of FL suffers from the skewed and biased non-IID local dataset acquired from the heterogeneous environment, which is common in real-world practice. To address this issue, we propose a novel Label-wise clustering algorithm in FL (FedLC) that guarantees the convergence among local clients that hold unique distribution. We theoretically analyze the non-IID attributes that potentially affect the performance, defining the six non-IID scenarios in hierarchical order. Through conducting experiments on the suggested non-IID scenarios, we empirically show that the vanilla FL aggregation model is incapable of gaining stable convergence, generating a biased model. By pre-evaluating the local dataset prior to training models, FedLC determines the potential contributions with respect to trainability in a global view, and adaptively selects the locals to collaboratively take part in while aggregation. Our experimental results show that the FedLC outperforms the state-of-the-art non-IID FL optimization studies, offering a robust convergence in a highly skewed and biased non-IID dataset.