Privacy-Preserving Asynchronous Vertical Federated Learning Algorithms for Multiparty Collaborative Learning

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 11; pp. 6103 - 6115
• Main Authors: Gu, Bin, Xu, An, Huo, Zhouyuan, Deng, Cheng, Huang, Heng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.11.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Asynchronous distributed computation; Collaboration; Collaborative work; Computer applications; Convergence; Convexity; Corporate learning; Data models; Data privacy; Distributed databases; Electronic commerce; Federated learning; Government industry relations; Learning algorithms; Machine learning; Partitioning algorithms; Privacy; privacy-preserving; stochastic gradient descent (SGD); Stochastic processes; Stochasticity; Trusted third parties; vertical federated learning
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3072238

The privacy-preserving federated learning for vertically partitioned (VP) data has shown promising results as the solution of the emerging multiparty joint modeling application, in which the data holders (such as government branches, private finance, and e-business companies) collaborate throughout the learning process rather than relying on a trusted third party to hold data. However, most of the existing federated learning algorithms for VP data are limited to synchronous computation. To improve the efficiency when the unbalanced computation/communication resources are common among the parties in the federated learning system, it is essential to develop asynchronous training algorithms for VP data while keeping the data privacy. In this article, we propose an asynchronous federated stochastic gradient descent (AFSGD-VP) algorithm and its two variance reduction variants, including stochastic variance reduced gradient (SVRG) and SAGA on the VP data. Moreover, we provide the convergence analyses of AFSGD-VP and its SVRG and SAGA variants under the condition of strong convexity and without any restrictions of staleness. We also discuss their model privacy, data privacy, computational complexities, and communication costs. To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for VP data with theoretical guarantees. Extensive experimental results on a variety of VP datasets not only verify the theoretical results of AFSGD-VP and its SVRG and SAGA variants but also show that our algorithms have much higher efficiency than the corresponding synchronous algorithms.