Regularized nonsmooth Newton method for multi-class support vector machines

• Published in: Optimization Methods and Software Vol. 22; no. 1; pp. 225 - 236
• Main Authors: Zhong, Ping, Fukushima, Masao
• Format: Journal Article
• Language: English; Japanese
• Published: Taylor & Francis 01.02.2007; Informa UK Limited
• Subjects: Box constrained variational inequality problem; D-gap function; Multi-class classification; Nonsmooth Newton method; Regularization methods
• ISSN: 1055-6788; 1029-4937
• DOI: 10.1080/10556780600834745

Multi-class classification is an important and on-going research subject in machine learning. Recently, the ν-K-SVCR method was proposed by the authors for multi-class classification. As many optimization problems have to be solved in multi-class classification, it is extremely important to develop an algorithm that can solve those optimization problems efficiently. In this article, the optimization problem in the ν-K-SVCR method is reformulated as an affine box constrained variational inequality problem with a positive semi-definite matrix, and a regularized version of the nonsmooth Newton method that uses the D-gap function as a merit function is applied to solve the resulting problems. The proposed algorithm fully exploits the typical feature of the ν-K-SVCR method, which enables us to reduce the size of Newton equations significantly. This indicates that the algorithm can be implemented efficiently in practice. The preliminary numerical experiments on benchmark data sets show that the proposed method is considerably faster than the standard Matlab routine used in the original ν-K-SVCR method.