A Unified Framework for Sparse Relaxed Regularized Regression: SR3

Regularized regression problems are ubiquitous in statistical modeling, signal processing, and machine learning. Sparse regression, in particular, has been instrumental in scientific model discovery, including compressed sensing applications, variable selection, and high-dimensional analysis. We pro...

Full description

Saved in:
Bibliographic Details
Published inIEEE access Vol. 7; pp. 1404 - 1423
Main Authors Zheng, Peng, Askham, Travis, Brunton, Steven L., Kutz, J. Nathan, Aravkin, Aleksandr Y.
Format Journal Article
LanguageEnglish
Published Piscataway IEEE 2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Algorithms
Compressed sensing
Convergence
Dimensional analysis
LASSO
Level set
Machine learning
matrix completion
Nonconvex optimization
Optimization
Regression analysis
Regularization
Signal processing
Signal processing algorithms
Sparse matrices
sparse regression
Sparsity
Statistical analysis
Statistical models
total variation regularization
Online AccessGet full text

Cover

More Information
Summary:Regularized regression problems are ubiquitous in statistical modeling, signal processing, and machine learning. Sparse regression, in particular, has been instrumental in scientific model discovery, including compressed sensing applications, variable selection, and high-dimensional analysis. We propose a broad framework for sparse relaxed regularized regression, called SR3. The key idea is to solve a relaxation of the regularized problem, which has three advantages over the state-of-the-art: 1) solutions of the relaxed problem are superior with respect to errors, false positives, and conditioning; 2) relaxation allows extremely fast algorithms for both convex and nonconvex formulations; and 3) the methods apply to composite regularizers, essential for total variation (TV) as well as sparsity-promoting formulations using tight frames. We demonstrate the advantages of SR3 (computational efficiency, higher accuracy, faster convergence rates, and greater flexibility) across a range of regularized regression problems with synthetic and real data, including applications in compressed sensing, LASSO, matrix completion, TV regularization, and group sparsity. Following standards of reproducible research, we also provide a companion MATLAB package that implements these examples.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2018.2886528