Bayesian Group Feature Selection for Support Vector Learning Machines

• Published in: Advances in Knowledge Discovery and Data Mining pp. 239 - 252
• Main Authors: Du, Changde, Du, Changying, Zhe, Shandian, Luo, Ali, He, Qing, Long, Guoping
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Data augmentation; Group feature selection; Support vector machine; Support vector regression; Variational bayesian inference
• ISBN: 3319317520; 9783319317526
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-31753-3_20

Group Feature Selection (GFS) has proven to be useful in improving the interpretability and prediction performance of learned model parameters in many machine learning and data mining applications. Existing GFS models were mainly based on square loss and logistic loss for regression and classification, leaving the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon $$\end{document}-insensitive loss and the hinge loss popularized by Support Vector Learning (SVL) machines still unexplored. In this paper, we present a Bayesian GFS framework for SVL machines based on the pseudo likelihood and data augmentation idea. With Bayesian inference, our method can circumvent the cross-validation for regularization parameters. Specifically, we apply the mean field variational method in an augmented space to derive the posterior distribution of model parameters and hyper-parameters for Bayesian estimation. Both regression and classification experiments conducted on synthetic and real-world data sets demonstrate that our proposed approach outperforms a number of competitors.