On Achieving Optimal Adversarial Test Error

We first elucidate various fundamental properties of optimal adversarial predictors: the structure of optimal adversarial convex predictors in terms of optimal adversarial zero-one predictors, bounds relating the adversarial convex loss to the adversarial zero-one loss, and the fact that continuous...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Li, Justin D, Matus Telgarsky
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 29.04.2024
Subjects
Errors
Perturbation
Training
Online AccessGet full text

Cover

More Information
Summary:We first elucidate various fundamental properties of optimal adversarial predictors: the structure of optimal adversarial convex predictors in terms of optimal adversarial zero-one predictors, bounds relating the adversarial convex loss to the adversarial zero-one loss, and the fact that continuous predictors can get arbitrarily close to the optimal adversarial error for both convex and zero-one losses. Applying these results along with new Rademacher complexity bounds for adversarial training near initialization, we prove that for general data distributions and perturbation sets, adversarial training on shallow networks with early stopping and an idealized optimal adversary is able to achieve optimal adversarial test error. By contrast, prior theoretical work either considered specialized data distributions or only provided training error guarantees.
ISSN:2331-8422