On the learning mechanism of adaptive filters

• Published in: IEEE transactions on signal processing Vol. 48; no. 6; pp. 1609 - 1625
• Main Authors: Nascimento, V.H., Sayed, A.H.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2000; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive filters; Adaptive systems; Analytical models; Chebyshev approximation; Convergence; Laboratories; Learning; Learning curves; Learning systems; Least squares approximation; Mathematical analysis; Signal processing; Simulation; Size measurement; Stability; Stability analysis; Studies
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/78.845919

This paper highlights, both analytically and by simulations, some interesting phenomena regarding the behavior of ensemble-average learning curves of adaptive filters that may have gone unnoticed. Among other results, the paper shows that even ensemble-average learning curves of single-tap LMS filters actually exhibit two distinct rates of convergence: one for the initial time instants and another, faster one, for later time instants. In addition, such curves tend to converge faster than predicted by mean-square theory and can converge even when a mean-square stability analysis predicts divergence. These effects tend to be magnified by increasing the step size. Two of the conclusions that follow from this work are (1) the mean-square stability alone may not be the most appropriate performance measure, especially for larger step sizes. A combination of mean-square stability and almost sure (a.s.) stability seems to be more appropriate. (2) Care is needed while interpreting ensemble-average curves for larger step sizes. The curves can lead to erroneous conclusions unless a large number of experiments are averaged (at times of the order of tens of thousands or higher).