Hyperparameter Search for Machine Learning Algorithms for Optimizing the Computational Complexity

For machine learning algorithms, fine-tuning hyperparameters is a computational challenge due to the large size of the problem space. An efficient strategy for adjusting hyperparameters can be established with the use of the greedy search and Swarm intelligence algorithms. The Random Search and Grid...

Full description

Saved in:
Bibliographic Details
Published inProcesses Vol. 11; no. 2; p. 349
Main Authors Ali, Yasser, Awwad, Emad, Al-Razgan, Muna, Maarouf, Ali
Format Journal Article
LanguageEnglish
Published Basel MDPI AG 01.02.2023
Subjects
Accuracy
Algorithms
Analysis
Artificial intelligence
Classification
Complexity
Computer applications
Computing costs
Data mining
Datasets
Genetic algorithms
Learning algorithms
Machine learning
Mathematical optimization
Methods
Neural networks
Optimization algorithms
Particle swarm optimization
Researchers
Search algorithms
Searches and seizures
Sensitivity analysis
Software
Support vector machines
Swarm intelligence
Online AccessGet full text

Cover

More Information
Summary:For machine learning algorithms, fine-tuning hyperparameters is a computational challenge due to the large size of the problem space. An efficient strategy for adjusting hyperparameters can be established with the use of the greedy search and Swarm intelligence algorithms. The Random Search and Grid Search optimization techniques show promise and efficiency for this task. The small population of solutions used at the outset, and the costly goal functions used by these searches, can lead to slow convergence or execution time in some cases. In this research, we propose using the machine learning model known as Support Vector Machine and optimizing it using four distinct algorithms—the Ant Bee Colony Algorithm, the Genetic Algorithm, the Whale Optimization, and the Particle Swarm Optimization—to evaluate the computational cost of SVM after hyper-tuning. Computational complexity comparisons of these optimization algorithms were performed to determine the most effective strategies for hyperparameter tuning. It was found that the Genetic Algorithm had a lower temporal complexity than other algorithms.
ISSN:2227-9717
2227-9717
DOI:10.3390/pr11020349