An interpretable regression approach based on bi-sparse optimization

Given the increasing amounts of data and high feature dimensionalities in forecasting problems, it is challenging to build regression models that are both computationally efficient and highly accurate. Moreover, regression models commonly suffer from low interpretability when using a single kernel f...

Full description

Saved in:
Bibliographic Details
Published inApplied intelligence (Dordrecht, Netherlands) Vol. 50; no. 11; pp. 4117 - 4142
Main Authors Zhang, Zhiwang, Gao, Guangxia, Yao, Tao, He, Jing, Tian, Yingjie
Format Journal Article
LanguageEnglish
Published New York Springer US 01.11.2020
Springer Nature B.V
Subjects
Algorithms
Artificial Intelligence
Computer Science
Kernel functions
Linear programming
Machines
Manufacturing
Mathematical analysis
Mechanical Engineering
Operators (mathematics)
Optimization
Processes
Regression models
Regularization
Regularization methods
Support vector machines
Support vector regression
Sparse learning
Zero-norm regularization
Data mining
Multi-kernel learning
Online AccessGet full text

Cover

More Information
Summary:Given the increasing amounts of data and high feature dimensionalities in forecasting problems, it is challenging to build regression models that are both computationally efficient and highly accurate. Moreover, regression models commonly suffer from low interpretability when using a single kernel function or a composite of multi-kernel functions to address nonlinear fitting problems. In this paper, we propose a bi-sparse optimization-based regression (BSOR) model and corresponding algorithm with reconstructed row and column kernel matrices in the framework of support vector regression (SVR). The BSOR model can predict continuous output values for given input points while using the zero-norm regularization method to achieve sparse instance and feature sets. Experiments were run on 16 datasets to compare BSOR to SVR, linear programming SVR (LPSVR), least squares SVR (LSSVR), multi-kernel learning SVR (MKLSVR), least absolute shrinkage and selection operator regression (LASSOR), and relevance vector regression (RVR). BSOR significantly outperformed the other six regression models in predictive accuracy, identification of the fewest representative instances, selection of the fewest important features, and interpretability of results, apart from its slightly high runtime.
ISSN:0924-669X
1573-7497
DOI:10.1007/s10489-020-01687-3