Speed up grid-search for parameter selection of support vector machines

• Published in: Applied soft computing Vol. 80; pp. 202 - 210
• Main Authors: Fayed, Hatem A., Atiya, Amir F.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.07.2019
• Subjects: Grid search; Parameter selection; Support vector machine; 90C20; 90C06; Grid search; 90C59; Parameter selection; 68W25; Support vector machine
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2019.03.037

Support vector machine (SVM) has been recently considered as one of the most efficient classifiers. However, the time complexity of kernel SVM, which is quadratic in the number of training patterns, makes it impractical to be applied to large data sets. In such a case, the complexity is further increased when an exhaustive grid search is used to find its optimal parameters (the kernel parameters and the penalty parameter, C). To reduce this extra complexity, a novel approach is proposed that prunes the data points by removing the ones that have extremely small chance of becoming support vectors. This is accomplished by using the support vectors obtained from the training of an SVM with a smaller value of C as the training patterns for an SVM with a slightly larger value. This can serve in reducing the grid-search time for the standard SVM and for the approximate methods that search heuristically for a small range of the kernel parameters first. Experiments showed the effectiveness of the proposed approach in reducing the training time for both methods considerably while achieving a similar accuracy to the standard SVM. In addition, the training time of the latter method was faster than that of the evolutionary techniques based on the particle swarm optimization algorithm. [Display omitted] •A speed-up exhaustive grid-search for parameter selection for SVM is achieved.•The proposed approach attained a similar accuracy to that of the standard SVM.•Combination of the proposed approach with ICDF speeds up the training considerably.•Parallel implementation of the proposed approach can also be performed as the standard SVM.