Unified LASSO Estimation by Least Squares Approximation

• Published in: Journal of the American Statistical Association Vol. 102; no. 479; pp. 1039 - 1048
• Main Authors: Wang, Hansheng, Leng, Chenlei
• Format: Journal Article
• Language: English
• Published: Alexandria Taylor & Francis 01.09.2007; American Statistical Association; Taylor & Francis Ltd
• Subjects: Adaptive LASSO; Algorithms; Approximation; Asymptotic methods; Bayes information criterion; Bayesian analysis; Covariance; Covariance matrices; Estimation; Estimation methods; Estimators; Generalized linear models; LASSO; Latent semantic analysis; Least angle regression; Least squares; Least squares approximation; Least squares method; Microarray data; Modeling; Objective functions; Oracle Property; Oracles; Regression analysis; Solution path; Statistical methods; Statistics; Theory and Methods
• ISSN: 0162-1459; 1537-274X
• DOI: 10.1198/016214507000000509

We propose a method of least squares approximation (LSA) for unified yet simple LASSO estimation. Our general theoretical framework includes ordinary least squares, generalized linear models, quantile regression, and many others as special cases. Specifically, LSA can transfer many different types of LASSO objective functions into their asymptotically equivalent least squares problems. Thereafter, the standard asymptotic theory can be established and the LARS algorithm can be applied. In particular, if the adaptive LASSO penalty and a Bayes information criterion-type tuning parameter selector are used, the resulting LSA estimator can be as efficient as the oracle. Extensive numerical studies confirm our theory.