A grammar-based GP approach applied to the design of deep neural networks

• Published in: Genetic programming and evolvable machines Vol. 23; no. 3; pp. 427 - 452
• Main Authors: Lima, Ricardo H. R., Magalhães, Dimmy, Pozo, Aurora, Mendiburu, Alexander, Santana, Roberto
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2022; Springer Nature B.V
• Subjects: Artificial Intelligence; Artificial neural networks; Biomedical Engineering and Bioengineering; Compilers; Computer Science; Configuration management; Electrical Engineering; Evolutionary algorithms; Genetic algorithms; Image classification; Image segmentation; Interpreters; Machine learning; Neural networks; Object recognition; Programming Languages; Programming Techniques; Software Engineering/Programming and Operating Systems; Special Issue: Highlights of Genetic Programming 2021 Events; Speech recognition; Grammatical evolution; Automatic design; Deep neural networks; Evolutionary algorithms; Genetic programming
• ISSN: 1389-2576; 1573-7632
• DOI: 10.1007/s10710-022-09432-0

Deep Learning has been very successful in automating the feature engineering process, widely applied for various tasks, such as speech recognition, classification, segmentation of images, time-series forecasting, among others. Deep neural networks (DNNs) incorporate the power to learn patterns through data, following an end-to-end fashion and expand the applicability in real world problems, since less pre-processing is necessary. With the fast growth in both scale and complexity, a new challenge has emerged regarding the design and configuration of DNNs. In this work, we present a study on applying an evolutionary grammar-based genetic programming algorithm (GP) as a unified approach to the design of DNNs. Evolutionary approaches have been growing in popularity for this subject as Neuroevolution is studied more. We validate our approach in three different applications: the design of Convolutional Neural Networks for image classification, Graph Neural Networks for text classification, and U-Nets for image segmentation. The results show that evolutionary grammar-based GP can efficiently generate different DNN architectures, adapted to each problem, employing choices that differ from what is usually seen in networks designed by hand. This approach has shown a lot of promise regarding the design of architectures, reaching competitive results with their counterparts.