Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent

This paper introduces Distributed Stein Variational Gradient Descent (DSVGD), a non-parametric generalized Bayesian inference framework for federated learning. DSVGD maintains a number of non-random and interacting particles at a central server to represent the current iterate of the model global po...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on signal processing Vol. 70; pp. 2180 - 2192
Main Authors Kassab, Rahif, Simeone, Osvaldo
Format Journal Article
LanguageEnglish
Published New York IEEE 2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Bayes methods
Bayesian analysis
bayesian learning
Collaborative work
Convergence
Federated learning
Free energy
Iterative methods
Machine learning
Servers
Signal processing algorithms
Statistical inference
Training
Uncertainty
variational inference
Online AccessGet full text

Cover

More Information
Summary:This paper introduces Distributed Stein Variational Gradient Descent (DSVGD), a non-parametric generalized Bayesian inference framework for federated learning. DSVGD maintains a number of non-random and interacting particles at a central server to represent the current iterate of the model global posterior. The particles are iteratively downloaded and updated by a subset of agents with the end goal of minimizing the global free energy. By varying the number of particles, DSVGD enables a flexible trade-off between per-iteration communication load and number of communication rounds. DSVGD is shown to compare favorably to benchmark frequentist and Bayesian federated learning strategies in terms of accuracy and scalability with respect to the number of agents, while also providing well-calibrated, and hence trustworthy, predictions.
ISSN:1053-587X
1941-0476
DOI:10.1109/TSP.2022.3168490