Differentially Private Byzantine-Robust Federated Learning

• Published in: IEEE transactions on parallel and distributed systems Vol. 33; no. 12; pp. 3690 - 3701
• Main Authors: Ma, Xu, Sun, Xiaoqian, Wu, Yuduo, Liu, Zheli, Chen, Xiaofeng, Dong, Changyu
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: byzantine-robust; Collaborative work; Computational modeling; Data models; Differential privacy; Federated learning; Machine learning; Privacy; Servers; Training
• ISSN: 1045-9219; 1558-2183
• DOI: 10.1109/TPDS.2022.3167434

Federated learning is a collaborative machine learning framework where a global model is trained by different organizations under the privacy restrictions. Promising as it is, privacy and robustness issues emerge when an adversary attempts to infer the private information from the exchanged parameters or compromise the global model. Various protocols have been proposed to counter the security risks, however, it becomes challenging when one wants to make federated learning protocols robust against Byzantine adversaries while preserving the privacy of the individual participant. In this article, we propose a differentially private Byzantine-robust federated learning scheme (DPBFL) with high computation and communication efficiency. The proposed scheme is effective in preventing adversarial attacks launched by the Byzantine participants and achieves differential privacy through a novel aggregation protocol in the shuffle model. The theoretical analysis indicates that the proposed scheme converges to the approximate optimal solution with the learning error dependent on the differential privacy budget and the number of Byzantine participants. Experimental results on MNIST, FashionMNIST and CIFAR10 demonstrate that the proposed scheme is effective and efficient.