Using the Strongest Adversarial Example to Alleviate Robust Overfitting

• Published in: Advanced Data Mining and Applications Vol. 13726; pp. 365 - 378
• Main Authors: Xu, Ce, Tang, Xiu, Lu, Peng
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer 2022; Springer Nature Switzerland
• Series: Lecture Notes in Computer Science
• Subjects: Adversarial training; Data augmentation; Robust overfitting
• ISBN: 3031221362; 9783031221361
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-031-22137-8_27

Overfitting is considered to be one of the dominant phenomena in machine learning. A recent study suggests that, just like standard training, adversarial training(AT) also suffers from the phenomenon of overfitting, which is named robust overfitting. It also points out that, among all the remedies for overfitting, early stopping seems to be the most effective way to alleviate it. In this paper, we explore the role of data augmentation in reducing robust overfitting. Inspired by MaxUp, we apply data augmentation to AT in a new way. The idea is to generate a set of augmented data and create adversarial examples(AEs) based on them. Then the strongest AE is applied to perform adversarial training. Combined with modern data augmentation techniques, we can simultaneously address the robust overfitting problem and improve the robust accuracy. Compared with previous research, our experiments show promising results on CIFAR-10 and CIFAR-100 datasets with PreactResnet18 model. Under the same condition, for l∞ $$\boldsymbol{l_\infty }$$ attack we boost the best robust accuracy by 1.57% $$\boldsymbol{1.57\%}$$ –2.89% $$\boldsymbol{2.89\%}$$ and the final robust accuracy by 7.51% $$\boldsymbol{7.51\%}$$ –9.42% $$\boldsymbol{9.42\%}$$ , for l2 $$\boldsymbol{l_2}$$ attack we improve the best robust accuracy by 1.64% $$\boldsymbol{1.64\%}$$ –1.74% $$\boldsymbol{1.74\%}$$ and the final robust accuracy by 3.80% $$\boldsymbol{3.80\%}$$ –5.99% $$\boldsymbol{5.99\%}$$ , respectively. Compared to other state-of-the-art models, our model also shows better results under the same experimental conditions. All codes for reproducing the experiments are available at https://github.com/xcfxr/adversarial_training.