Universal Weighted MSE Improvement of the Least-Squares Estimator

• Published in: IEEE transactions on signal processing Vol. 56; no. 5; pp. 1788 - 1800
• Main Author: Eldar, Y.C.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.05.2008; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Admissible estimators; Applied sciences; Communication system control; Context; Covariance matrix; dominating estimators; Estimation error; Estimators; Exact sciences and technology; Gaussian processes; Information, signal and communications theory; Least squares method; linear estimation; Linear regression; Mathematical analysis; Mathematical models; Matrices; Minimax techniques; Miscellaneous; Norms; Parameter estimation; Regression; Regression analysis; Seismology; Signal processing; Strategy; Studies; Telecommunications and information theory; Weight measurement; weighted minimax MSE estimation; Performance evaluation; dominating estimators; Linear regression; Worst case method; Regression analysis; Linear model; Mean square error; Weighting; Regression model; Least squares method; weighted minimax MSE estimation; Minimax method; Signal processing; Linear estimation; Admissible estimators
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2007.913158

Since the seminal work of Stein in the 1950s, there has been continuing research devoted to improving the total mean-squared error (MSE) of the least-squares (LS) estimator in the linear regression model. However, a drawback of these methods is that although they improve the total MSE, they do so at the expense of increasing the MSE of some of the individual signal components. Here we consider a framework for developing linear estimators that outperform the LS strategy over bounded norm signals, under all weighted MSE measures. This guarantees, for example, that both the total MSE and the MSE of each of the elements will be smaller than that resulting from the LS approach. We begin by deriving an easily verifiable condition on a linear method that ensures LS domination for every weighted MSE. We then suggest a minimax estimator that minimizes the worst-case MSE over all weighting matrices and bounded norm signals subject to the universal weighted MSE domination constraint.