An efficient optimization approach for designing machine learning models based on genetic algorithm

• Published in: Neural computing & applications Vol. 33; no. 6; pp. 1923 - 1933
• Main Authors: Hamdia, Khader M., Zhuang, Xiaoying, Rabczuk, Timon
• Format: Journal Article
• Language: English
• Published: London Springer London 01.03.2021; Springer Nature B.V
• Subjects: Adaptive systems; Artificial Intelligence; Artificial neural networks; Complex systems; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Design optimization; Error analysis; Fuzzy logic; Genetic algorithms; Image Processing and Computer Vision; Machine learning; Model accuracy; Nanoparticles; Optimization; Original Article; Polymer matrix composites; Probability and Statistics in Computer Science; Deep neural networks; Genetic algorithm; Fracture energy; Optimization; Polymer nanocomposites; Machine learning
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-020-05035-x

Machine learning (ML) methods have shown powerful performance in different application. Nonetheless, designing ML models remains a challenge and requires further research as most procedures adopt a trial and error strategy. In this study, we present a methodology to optimize the architecture and the feature configurations of ML models considering a supervised learning process. The proposed approach employs genetic algorithm (GA)-based integer-valued optimization for two ML models, namely deep neural networks (DNN) and adaptive neuro-fuzzy inference system (ANFIS). The selected variables in the DNN optimization problems are the number of hidden layers, their number of neurons and their activation function, while the type and the number of membership functions are the design variables in the ANFIS optimization problem. The mean squared error (MSE) between the predictions and the target outputs is minimized as the optimization fitness function. The proposed scheme is validated through a case study of computational material design. We apply the method to predict the fracture energy of polymer/nanoparticles composites (PNCs) with a database gathered from the literature. The optimized DNN model shows superior prediction accuracy compared to the classical one-hidden layer network. Also, it outperforms ANFIS with significantly lower number of generations in GA. The proposed method can be easily extended to optimize similar architecture properties of ML models in various complex systems.