Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks
One major goal of the AI security community is to securely and reliably produce and deploy deep learning models for real-world applications. To this end, data poisoning based backdoor attacks on deep neural networks (DNNs) in the production stage (or training stage) and corresponding defenses are ex...
Saved in:
Main Authors | , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
25.11.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | One major goal of the AI security community is to securely and reliably
produce and deploy deep learning models for real-world applications. To this
end, data poisoning based backdoor attacks on deep neural networks (DNNs) in
the production stage (or training stage) and corresponding defenses are
extensively explored in recent years. Ironically, backdoor attacks in the
deployment stage, which can often happen in unprofessional users' devices and
are thus arguably far more threatening in real-world scenarios, draw much less
attention of the community. We attribute this imbalance of vigilance to the
weak practicality of existing deployment-stage backdoor attack algorithms and
the insufficiency of real-world attack demonstrations. To fill the blank, in
this work, we study the realistic threat of deployment-stage backdoor attacks
on DNNs. We base our study on a commonly used deployment-stage attack paradigm
-- adversarial weight attack, where adversaries selectively modify model
weights to embed backdoor into deployed DNNs. To approach realistic
practicality, we propose the first gray-box and physically realizable weights
attack algorithm for backdoor injection, namely subnet replacement attack
(SRA), which only requires architecture information of the victim model and can
support physical triggers in the real world. Extensive experimental simulations
and system-level real-world attack demonstrations are conducted. Our results
not only suggest the effectiveness and practicality of the proposed attack
algorithm, but also reveal the practical risk of a novel type of computer virus
that may widely spread and stealthily inject backdoor into DNN models in user
devices. By our study, we call for more attention to the vulnerability of DNNs
in the deployment stage. |
---|---|
DOI: | 10.48550/arxiv.2111.12965 |