Universal learning using free multivariate splines

• Published in: Neurocomputing (Amsterdam) Vol. 119; pp. 253 - 263
• Main Authors: Lei, Yunwen, Ding, Lixin, Wu, Weili
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 07.11.2013; Elsevier
• Subjects: Applied sciences; Approximation; Artificial intelligence; Complexity regularization; Computer science; control theory; systems; Exact sciences and technology; Free multivariate spline; Learning theory; Rademacher complexity; Learning theory; Free multivariate spline; Rademacher complexity; Complexity regularization; Estimation error; Free field; Error estimation; Logarithmic function; Sample size; Bounded error; Error bound; Risk analysis; Generalization; Computational complexity; Modeling; Spline approximation; Loss function; Space complexity; Approximation error; Learning algorithm; Artificial intelligence; Regularity
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2013.03.033

This paper discusses the problem of universal learning using free multivariate splines of order 1. Universal means that the learning algorithm does not involve a priori assumption on the regularity of the target function. We characterize the complexity of the space of free multivariate splines by the remarkable notion called Rademacher complexity, based on which a penalized empirical risk is constructed as an estimation of the expected risk for the candidate model. Our Rademacher complexity bounds are tight within a logarithmic factor. It is shown that the prediction rule minimizing the penalized empirical risk achieves a favorable balance between the approximation and estimation error. By resorting to the powerful techniques in approximation theory to approach the approximation error, we also derive bounds on the generalization error in terms of the sample size, for a large class of loss functions.