Sequential Projection Pursuit with Kernel Matrix Update and Symbolic Model Selection

• Published in: IEEE transactions on cybernetics Vol. 44; no. 12; pp. 2458 - 2469
• Main Authors: Rodriguez-Martinez, E., Mu, T., Goulermas, J. Y.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2014; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Benchmarking; Covariance matrices; Cybernetics; Evolutionary; Evolutionary optimization; Feature extraction; Indexes; Kernel; kernel matrix update; Kernels; Optimization; Projection; projection pursuit; Searching; supervised dimensionality reduction; Training; Vectors; kernel matrix update; supervised dimensionality reduction; projection pursuit; Evolutionary optimization
• ISSN: 2168-2267; 2168-2275
• DOI: 10.1109/TCYB.2014.2308908

This paper proposes a novel way for generating reliable low-dimensional features with improved class separability in a kernel-induced feature space. The feature projections rely on a very efficient sequential projection pursuit method, adapted to support nonlinear projections using a new kernel matrix update scheme. This enables the gradual removal of structure from the space of residual dimensions to allow the recovery of multiple projections. An adaptive kernel function is employed to unfold different types of data characteristics. We follow a holistic model selection procedure that, together with the optimal projections, dimensionality, and kernel parameters, additionally optimizes symbolically the projection index that controls the actual measurement of the data interestingness without user interaction. We tackle the underlying complex bi-level optimization model as a mixture of evolutionary and gradient search. The effectiveness of the proposed algorithm over existing approaches is demonstrated with benchmark evaluations and comparisons.