Combining meta-learning and search techniques to select parameters for support vector machines

• Published in: Neurocomputing (Amsterdam) Vol. 75; no. 1; pp. 3 - 13
• Main Authors: Gomes, Taciana A.F., Prudêncio, Ricardo B.C., Soares, Carlos, Rossi, André L.D., Carvalho, André
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 2012
• Subjects: Kernels; Learning; Meta-learning; Optimization; Regression; Regularization; Search; Search algorithms; Searching; Support vector machines; Support vector machines; Meta-learning; Search
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2011.07.005

Support Vector Machines (SVMs) have achieved very good performance on different learning problems. However, the success of SVMs depends on the adequate choice of the values of a number of parameters (e.g., the kernel and regularization parameters). In the current work, we propose the combination of meta-learning and search algorithms to deal with the problem of SVM parameter selection. In this combination, given a new problem to be solved, meta-learning is employed to recommend SVM parameter values based on parameter configurations that have been successfully adopted in previous similar problems. The parameter values returned by meta-learning are then used as initial search points by a search technique, which will further explore the parameter space. In this proposal, we envisioned that the initial solutions provided by meta-learning are located in good regions of the search space (i.e. they are closer to optimum solutions). Hence, the search algorithm would need to evaluate a lower number of candidate solutions when looking for an adequate solution. In this work, we investigate the combination of meta-learning with two search algorithms: Particle Swarm Optimization and Tabu Search. The implemented hybrid algorithms were used to select the values of two SVM parameters in the regression domain. These combinations were compared with the use of the search algorithms without meta-learning. The experimental results on a set of 40 regression problems showed that, on average, the proposed hybrid methods obtained lower error rates when compared to their components applied in isolation.