Sharpness-Aware Minimization Alone can Improve Adversarial Robustness
Sharpness-Aware Minimization (SAM) is an effective method for improving generalization ability by regularizing loss sharpness. In this paper, we explore SAM in the context of adversarial robustness. We find that using only SAM can achieve superior adversarial robustness without sacrificing clean acc...
Saved in:
Main Authors | , , |
---|---|
Format | Journal Article |
Language | English |
Published |
09.05.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Sharpness-Aware Minimization (SAM) is an effective method for improving
generalization ability by regularizing loss sharpness. In this paper, we
explore SAM in the context of adversarial robustness. We find that using only
SAM can achieve superior adversarial robustness without sacrificing clean
accuracy compared to standard training, which is an unexpected benefit. We also
discuss the relation between SAM and adversarial training (AT), a popular
method for improving the adversarial robustness of DNNs. In particular, we show
that SAM and AT differ in terms of perturbation strength, leading to different
accuracy and robustness trade-offs. We provide theoretical evidence for these
claims in a simplified model. Finally, while AT suffers from decreased clean
accuracy and computational overhead, we suggest that SAM can be regarded as a
lightweight substitute for AT under certain requirements. Code is available at
https://github.com/weizeming/SAM_AT. |
---|---|
DOI: | 10.48550/arxiv.2305.05392 |