Variants of Non-Negative Least-Mean-Square Algorithm and Convergence Analysis

• Published in: IEEE transactions on signal processing Vol. 62; no. 15; pp. 3990 - 4005
• Main Authors: Jie Chen, Richard, Cedric, Bermudez, Jose-Carlos M., Honeine, Paul
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.08.2014; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive algorithms; Adaptive signal processing; Algorithm design and analysis; Algorithms; Applied sciences; Behavior; Computational efficiency; Computer Science; Convergence; convergence analysis; Detection, estimation, filtering, equalization, prediction; Economic models; Engineering Sciences; Equations; Estimation; Exact sciences and technology; exponential algorithm; Gaussian; Information, signal and communications theory; least-mean-square algorithms; Machine Learning; Mathematical analysis; Mathematical models; non-negativity constraints; normalized algorithm; Prediction algorithms; sign-sign algorithm; Signal and communications theory; Signal and Image processing; Signal processing algorithms; Signal, noise; Telecommunications and information theory; Transaction processing; Vectors; Performance evaluation; Second order; Parameter estimation; Wiener filtering; Input signal; non-negativity constraints; least-mean-square algorithms; Adaptive signal processing; Algorithm; Computational complexity; Adaptive method; exponential algorithm; sign-sign algorithm; Algorithm performance; Simulation; convergence analysis; First order; Analytical method; Convergence rate; Signal processing; normalized algorithm; Algorithm analysis; Least mean squares methods; Algorithm design and analysis; least mean squares methods; nonstationary environments; Wiener filters; non-negativity; nonnegative least-mean-square algorithm; Convergence; Wiener filtering problem; parameter estimation; Prediction algorithms; Gaussian inputs; stochastic processes; NNLMS; adaptive filtering; computational cost; stochastic behavior; Estimation; adaptive weights; Vectors; Equations; Signal processing algorithms
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2014.2332440

Due to the inherent physical characteristics of systems under investigation, non-negativity is one of the most interesting constraints that can usually be imposed on the parameters to estimate. The Non-Negative Least-Mean-Square algorithm (NNLMS) was proposed to adaptively find solutions of a typical Wiener filtering problem but with the side constraint that the resulting weights need to be non-negative. It has been shown to have good convergence properties. Nevertheless, certain practical applications may benefit from the use of modified versions of this algorithm. In this paper, we derive three variants of NNLMS. Each variant aims at improving the NNLMS performance regarding one of the following aspects: sensitivity of input power, unbalance of convergence rates for different weights and computational cost. We study the stochastic behavior of the adaptive weights for these three new algorithms for non-stationary environments. This study leads to analytical models to predict the first and second order moment behaviors of the weights for Gaussian inputs. Simulation results are presented to illustrate the performance of the new algorithms and the accuracy of the derived models.