Analysing hyper-heuristics based on Neural Networks for the automatic design of population-based metaheuristics in continuous optimisation problems

• Published in: Swarm and evolutionary computation Vol. 89; p. 101616
• Main Authors: Tapia-Avitia, José M., Cruz-Duarte, Jorge M., Amaya, Ivan, Ortiz-Bayliss, José Carlos, Terashima-Marin, Hugo, Pillay, Nelishia
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.2024
• Subjects: Hyper-heuristics; Metaheuristics; Neural Networks; Parameter tuning and algorithm configuration; Performance measures; Neural Networks; Performance measures; Hyper-heuristics; Parameter tuning and algorithm configuration; Metaheuristics
• ISSN: 2210-6502
• DOI: 10.1016/j.swevo.2024.101616

When dealing with optimisation problems, Metaheuristics (MHs) quickly come to our minds. A quick literature review reveals a vast universe of MHs. Although the metaphors behind these MHs are always presented as ‘unique’ to justify their novelty, the truth is that many MHs just recombine elements from other techniques. Then, instead of proposing MHs based on what already exists in nature, it is better to follow a standard model for the automatic metaheuristic design by employing simple heuristics. Many approaches have designed algorithms that probe the combination of such heuristics, generating astonishing results compared to generic MHs. Following this idea, our work examines Neural Network (NN) architectures over several control variables to tailor MHs. Our results render an architecture that enhances the results compared to generic MHs at 91%, those MHs produced by Random Search at 81%, and the current state-of-the-art NN model at 66%. We notice a big gap for NN-based models with different architectures, which are worth investigating. Among the benefits of our proposed approach is that it reduces the dependence on human knowledge, moving towards the automatic generation of solving methods that learn from empirical data how to succeed in various continuous optimisation scenarios. [Display omitted] •Neural network architectures as hyper-heuristics for automatic metaheuristic design.•Our model beats 91% of known metaheuristics and 66% state-of-the-art hyper-heuristics.•We identify different approaches based on computing burden and memory constraints.•We proof that neural networks with different architectures are worth investigating.