Distributed Kernel-Based Gradient Descent Algorithms

We study the generalization ability of distributed learning equipped with a divide-and-conquer approach and gradient descent algorithm in a reproducing kernel Hilbert space (RKHS). Using special spectral features of the gradient descent algorithms and a novel integral operator approach, we provide o...

Full description

Saved in:
Bibliographic Details
Published inConstructive approximation Vol. 47; no. 2; pp. 249 - 276
Main Authors Lin, Shao-Bo, Zhou, Ding-Xuan
Format Journal Article
LanguageEnglish
Published New York Springer US 01.04.2018
Springer Nature B.V
Subjects
Algorithms
Analysis
Hilbert space
Machine learning
Mathematics
Mathematics and Statistics
Numerical Analysis
Operators (mathematics)
Statistical analysis
Integral operator
41A35
Distributed learning
68T05
Gradient descent algorithm
Learning theory
94A20
Online AccessGet full text

Cover

More Information
Summary:We study the generalization ability of distributed learning equipped with a divide-and-conquer approach and gradient descent algorithm in a reproducing kernel Hilbert space (RKHS). Using special spectral features of the gradient descent algorithms and a novel integral operator approach, we provide optimal learning rates of distributed gradient descent algorithms in probability and partly conquer the saturation phenomenon in the literature in the sense that the maximum number of local machines to guarantee the optimal learning rates does not vary if the regularity of the regression function goes beyond a certain quantity. We also find that additional unlabeled data can help relax the restriction on the number of local machines in distributed learning.
ISSN:0176-4276
1432-0940
DOI:10.1007/s00365-017-9379-1