Stochastic Behavior Analysis of the Gaussian Kernel Least-Mean-Square Algorithm

• Published in: IEEE transactions on signal processing Vol. 60; no. 5; pp. 2208 - 2222
• Main Authors: Parreira, W. D., Bermudez, J. C. M., Richard, C., Tourneret, Jean-Yves
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.05.2012; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive filtering; Adaptive filters; Algorithm design and analysis; Algorithms; Applied sciences; Artificial Intelligence; Computer Science; Computer simulation; Computer Vision and Pattern Recognition; convergence analysis; Correlation; Detection, estimation, filtering, equalization, prediction; Dictionaries; Exact sciences and technology; Gaussian; Graphics; Image Processing; Information, signal and communications theory; Kernel; kernel least-mean-square (KLMS); Kernels; Mathematical analysis; Mathematical models; nonlinear system; Nonlinearity; Optimized production technology; reproducing kernel; Signal and communications theory; Signal and Image Processing; Signal processing algorithms; Signal, noise; Studies; Telecommunications and information theory; Vectors; Adaptive filtering; Unsteady state; nonlinear system; Non linear phenomenon; kernel least-mean-square (KLMS); Implementation; Mean square error; Convergence rate; Robustness; Monte Carlo method; Design criterion; Adaptive filter; reproducing kernel; Algorithm; Steady state; Non linear filtering; Kernel method; Stochastic analysis; convergence analysis; Linear filter; Analytical method; A priori estimation; Recursive method; Signal processing; Numerical simulation; Non linear effect; Least mean squares methods; Reproducing kernel; Kernel least-mean-square; KLMS; Nonlinear system; Convergence analysis
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2012.2186132

The kernel least-mean-square (KLMS) algorithm is a popular algorithm in nonlinear adaptive filtering due to its simplicity and robustness. In kernel adaptive filters, the statistics of the input to the linear filter depends on the parameters of the kernel employed. Moreover, practical implementations require a finite nonlinearity model order. A Gaussian KLMS has two design parameters, the step size and the Gaussian kernel bandwidth. Thus, its design requires analytical models for the algorithm behavior as a function of these two parameters. This paper studies the steady-state behavior and the transient behavior of the Gaussian KLMS algorithm for Gaussian inputs and a finite order nonlinearity model. In particular, we derive recursive expressions for the mean-weight-error vector and the mean-square-error. The model predictions show excellent agreement with Monte Carlo simulations in transient and steady state. This allows the explicit analytical determination of stability limits, and gives opportunity to choose the algorithm parameters a priori in order to achieve prescribed convergence speed and quality of the estimate. Design examples are presented which validate the theoretical analysis and illustrates its application.