Out-of-Sample Extensions for Non-Parametric Kernel Methods

• Published in: IEEE transaction on neural networks and learning systems Vol. 28; no. 2; pp. 334 - 345
• Main Authors: Pan, Binbin, Chen, Wen-Sheng, Chen, Bo, Xu, Chen, Lai, Jianhuang
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.02.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Data points; Estimating techniques; Face recognition; Geometry; Hilbert space; Hyperkernels; inductive learning; Information processing; Kernel; Kernel functions; kernel methods; Learning; Manifolds; nonparametric kernel learning; Nonparametric statistics; out-of-sample extensions; Pattern recognition; Regression analysis; Risk management; State of the art; Support vector machines
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2015.2512277

Choosing suitable kernels plays an important role in the performance of kernel methods. Recently, a number of studies were devoted to developing nonparametric kernels. Without assuming any parametric form of the target kernel, nonparametric kernel learning offers a flexible scheme to utilize the information of the data, which may potentially characterize the data similarity better. The kernel methods using nonparametric kernels are referred to as nonparametric kernel methods. However, many nonparametric kernel methods are restricted to transductive learning, where the prediction function is defined only over the data points given beforehand. They have no straightforward extension for the out-of-sample data points, and thus cannot be applied to inductive learning. In this paper, we show how to make the nonparametric kernel methods applicable to inductive learning. The key problem of out-of-sample extension is how to extend the nonparametric kernel matrix to the corresponding kernel function. A regression approach in the hyper reproducing kernel Hilbert space is proposed to solve this problem. Empirical results indicate that the out-of-sample performance is comparable to the in-sample performance in most cases. Experiments on face recognition demonstrate the superiority of our nonparametric kernel method over the state-of-the-art parametric kernel methods.