MM for penalized estimation

Penalized estimation can conduct variable selection and parameter estimation simultaneously. The general framework is to minimize a loss function subject to a penalty designed to generate sparse variable selection. The majorization–minimization (MM) algorithm is a computational scheme for stability...

Full description

Saved in:
Bibliographic Details
Published inTest (Madrid, Spain) Vol. 31; no. 1; pp. 54 - 75
Main Author Wang, Zhu
Format Journal Article
LanguageEnglish
Published Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2022
Springer Nature B.V
Subjects
Algorithms
Economics
Feature selection
Finance
Generalized linear models
Insurance
Machine learning
Management
Mathematics and Statistics
Optimization
Original Paper
Outliers (statistics)
Parameter estimation
Robustness
Statistical models
Statistical Theory and Methods
Statistics
Statistics for Business
62J07
Classification
68U01
MM algorithm
Regression
Quadratic majorization
62H30
62-08
Robust estimation
62F35
Nonconvex
Variable selection
Online AccessGet full text

Cover

More Information
Summary:Penalized estimation can conduct variable selection and parameter estimation simultaneously. The general framework is to minimize a loss function subject to a penalty designed to generate sparse variable selection. The majorization–minimization (MM) algorithm is a computational scheme for stability and simplicity, and the MM algorithm has been widely applied in penalized estimation. Much of the previous work has focused on convex loss functions such as generalized linear models. When data are contaminated with outliers, robust loss functions can generate more reliable estimates. Recent literature has witnessed a growing impact of nonconvex loss-based methods, which can generate robust estimation for data contaminated with outliers. This article investigates MM algorithm for penalized estimation, provides innovative optimality conditions and establishes convergence theory with both convex and nonconvex loss functions. With respect to applications, we focus on several nonconvex loss functions, which were formerly studied in machine learning for regression and classification problems. Performance of the proposed algorithms is evaluated on simulated and real data including cancer clinical status. Efficient implementations of the algorithms are available in the R package mpath in CRAN.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1133-0686
1863-8260
DOI:10.1007/s11749-021-00770-2