Fast Kernel-Based Independent Component Analysis

• Published in: IEEE transactions on signal processing Vol. 57; no. 9; pp. 3498 - 3511
• Main Authors: Hao Shen, Jegelka, S., Gretton, A.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.09.2009; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Approximate Newton-like methods; Computational efficiency; Criteria; Cybernetics; Estimates; Exact sciences and technology; Extraterrestrial measurements; Hilbert-Schmidt independence criterion; Independent component analysis; independent component analysis (ICA); Information, signal and communications theory; Kernel; kernel methods; Kernels; Machine learning; Mathematical models; Miscellaneous; Mutual information; Optimization; Optimization methods; orthogonal group; Parametric statistics; Particle measurements; Signal processing; Studies; Telecommunications and information theory; Performance evaluation; Approximate method; Independent component analysis; Matrix decomposition; Algorithm; Hilbert-Schmidt independence criterion; Optimization; Kernel method; Outlier; Kernel function; Accuracy; Flattening; Cholesky method; independent component analysis (ICA); Descent method; Signal processing; kernel methods; Approximate Newton-like methods; Open market; Newton method; Gradient method; orthogonal group
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2009.2022857

Recent approaches to independent component analysis (ICA) have used kernel independence measures to obtain highly accurate solutions, particularly where classical methods experience difficulty (for instance, sources with near-zero kurtosis). FastKICA (fast HSIC-based kernel ICA) is a new optimization method for one such kernel independence measure, the Hilbert-Schmidt Independence Criterion (HSIC). The high computational efficiency of this approach is achieved by combining geometric optimization techniques, specifically an approximate Newton-like method on the orthogonal group, with accurate estimates of the gradient and Hessian based on an incomplete Cholesky decomposition. In contrast to other efficient kernel-based ICA algorithms, FastKICA is applicable to any twice differentiable kernel function. Experimental results for problems with large numbers of sources and observations indicate that FastKICA provides more accurate solutions at a given cost than gradient descent on HSIC. Comparing with other recently published ICA methods, FastKICA is competitive in terms of accuracy, relatively insensitive to local minima when initialized far from independence, and more robust towards outliers. An analysis of the local convergence properties of FastKICA is provided.