Effects of Random Sampling on SVM Hyper-parameter Tuning

• Published in: Intelligent Systems Design and Applications Vol. 557; pp. 268 - 278
• Main Authors: Horváth, Tomáš, Mantovani, Rafael G., de Carvalho, André C. P. L. F.
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Advances in Intelligent Systems and Computing
• Subjects: Artificial intelligence; Hyper-parameter tuning; Sampling; Suport vector machines
• ISBN: 9783319534794; 3319534793
• ISSN: 2194-5357; 2194-5365
• DOI: 10.1007/978-3-319-53480-0_27

Hyper-parameter tuning is one of the crucial steps in the successful application of machine learning algorithms to real data. In general, the tuning process is modeled as an optimization problem for which several methods have been proposed. For complex algorithms, the evaluation of a hyper-parameter configuration is expensive and their runtime is speed up through data sampling. In this paper, the effect of sample sizes to the results of hyper-parameter tuning process is investigated. Hyper-parameters of Support Vector Machines are tuned on samples of different sizes generated from a dataset. Hausdorff distance is proposed for computing the differences between the results of hyper-parameter tuning on two samples of different size. 100 real-world datasets and two tuning methods (Random Search and Particle Swarm Optimization) are used in the experiments revealing some interesting relations between sample sizes and results of hyper-parameter tuning which open some promising directions for future investigation in this direction.