FedPHP: Federated Personalization with Inherited Private Models

• Published in: Machine Learning and Knowledge Discovery in Databases. Research Track Vol. 12975; pp. 587 - 602
• Main Authors: Li, Xin-Chun, Zhan, De-Chuan, Shao, Yunfeng, Li, Bingshuai, Song, Shaoming
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• ISBN: 3030864855; 9783030864859
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-86486-6_36

Federated Learning (FL) generates a single global model via collaborating distributed clients without leaking data privacy. However, the statistical heterogeneity of non-iid data across clients poses a fundamental challenge to the model personalization process of each client. Our significant observation is that the newly downloaded global model from the server may perform poorly on local clients, while it could become better after adequate personalization steps. Inspired by this, we advocate that the hard-won personalized model in each communication round should be rationally exploited, while standard FL methods directly overwrite the previous personalized models. Specifically, we propose a novel concept named “inHerited Private Model” (HPM) for each local client as a temporal ensembling of its historical personalized models and exploit it to supervise the personalization process in the next global round. We explore various types of knowledge transfer to facilitate the personalization process. We provide both theoretical analysis and abundant experimental studies to verify the superiorities of our algorithm.