Robustness Enhancement of Neural Networks via Architecture Search with Multi-Objective Evolutionary Optimization

• Published in: Mathematics (Basel) Vol. 10; no. 15; p. 2724
• Main Authors: Chen, Haojie, Huang, Hai, Zuo, Xingquan, Zhao, Xinchao
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.08.2022
• Subjects: Accuracy; adversarial defense; CLEVER score; Computer architecture; Deep learning; Design; Elitism; Evolutionary algorithms; Genetic algorithms; Machine learning; Mathematics; Methods; Multiple objective analysis; neural architecture search; Neural networks; Objectives; Optimization; Optimization algorithms; Performance evaluation; Population; Robustness; Search algorithms; surrogate model
• ISSN: 2227-7390; 2227-7390
• DOI: 10.3390/math10152724

Along with the wide use of deep learning technology, its security issues have drawn much attention over the years. Adversarial examples expose the inherent vulnerability of deep learning models and make it a challenging task to improve their robustness. Model robustness is related not only to its parameters but also to its architecture. This paper proposes a novel robustness enhanced approach for neural networks based on a neural architecture search. First, we randomly sample multiple neural networks to construct the initial population. Second, we utilize the individual networks in the population to fit and update the surrogate models. Third, the population of neural networks is evolved through a multi-objective evolutionary algorithm, where the surrogate models accelerate the performance evaluation of networks. Finally, the second and third steps are performed alternately until a network architecture with high accuracy and robustness is achieved. Experimental results show that the proposed method outperforms some classical artificially designed neural networks and other architecture search algorithms in terms of robustness.