The Effect of Personalization in FedProx: A Fine-grained Analysis on Statistical Accuracy and Communication Efficiency

FedProx is a simple yet effective federated learning method that enables model personalization via regularization. Despite remarkable success in practice, a rigorous analysis of how such a regularization provably improves the statistical accuracy of each client's local model hasn't been fu...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Yu, Xin, Zelin He, Sun, Ying, Xue, Lingzhou, Li, Runze
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 11.10.2024
Subjects
Accuracy
Algorithms
Cost analysis
Customization
Federated learning
Heterogeneity
Regularization
Resource allocation
Online AccessGet full text

Cover

More Information
Summary:FedProx is a simple yet effective federated learning method that enables model personalization via regularization. Despite remarkable success in practice, a rigorous analysis of how such a regularization provably improves the statistical accuracy of each client's local model hasn't been fully established. Setting the regularization strength heuristically presents a risk, as an inappropriate choice may even degrade accuracy. This work fills in the gap by analyzing the effect of regularization on statistical accuracy, thereby providing a theoretical guideline for setting the regularization strength for achieving personalization. We prove that by adaptively choosing the regularization strength under different statistical heterogeneity, FedProx can consistently outperform pure local training and achieve a nearly minimax-optimal statistical rate. In addition, to shed light on resource allocation, we design an algorithm, provably showing that stronger personalization reduces communication complexity without increasing the computation cost overhead. Finally, our theory is validated on both synthetic and real-world datasets and its generalizability is verified in a non-convex setting.
ISSN:2331-8422