Convex Formulation for Kernel PCA and Its Use in Semisupervised Learning

• Published in: IEEE transaction on neural networks and learning systems Vol. 29; no. 8; pp. 3863 - 3869
• Main Authors: Alaiz, Carlos M., Fanuel, Michael, Suykens, Johan A. K.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Computational geometry; Convex analysis; Convex functions; Convexity; Eigenvalues and eigenfunctions; Kernel; Kernel principal component analysis (KPCA); kernel spectral clustering; Lagrange multiplier; Mathematical models; Optimization; Principal component analysis; Principal components analysis; Regularization; Saddle points; Semi-supervised learning; Semisupervised learning; Support vector machines
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2017.2709838

In this brief, kernel principal component analysis (KPCA) is reinterpreted as the solution to a convex optimization problem. Actually, there is a constrained convex problem for each principal component, so that the constraints guarantee that the principal component is indeed a solution, and not a mere saddle point. Although these insights do not imply any algorithmic improvement, they can be used to further understand the method, formulate possible extensions, and properly address them. As an example, a new convex optimization problem for semisupervised classification is proposed, which seems particularly well suited whenever the number of known labels is small. Our formulation resembles a least squares support vector machine problem with a regularization parameter multiplied by a negative sign, combined with a variational principle for KPCA. Our primal optimization principle for semisupervised learning is solved in terms of the Lagrange multipliers. Numerical experiments in several classification tasks illustrate the performance of the proposed model in problems with only a few labeled data.