Formalizing and Estimating Distribution Inference Risks
Distribution inference, sometimes called property inference, infers statistical properties about a training set from access to a model trained on that data. Distribution inference attacks can pose serious risks when models are trained on private data, but are difficult to distinguish from the intrin...
Saved in:
Main Authors | , |
---|---|
Format | Journal Article |
Language | English |
Published |
13.09.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Distribution inference, sometimes called property inference, infers
statistical properties about a training set from access to a model trained on
that data. Distribution inference attacks can pose serious risks when models
are trained on private data, but are difficult to distinguish from the
intrinsic purpose of statistical machine learning -- namely, to produce models
that capture statistical properties about a distribution. Motivated by Yeom et
al.'s membership inference framework, we propose a formal definition of
distribution inference attacks that is general enough to describe a broad class
of attacks distinguishing between possible training distributions. We show how
our definition captures previous ratio-based property inference attacks as well
as new kinds of attack including revealing the average node degree or
clustering coefficient of a training graph. To understand distribution
inference risks, we introduce a metric that quantifies observed leakage by
relating it to the leakage that would occur if samples from the training
distribution were provided directly to the adversary. We report on a series of
experiments across a range of different distributions using both novel
black-box attacks and improved versions of the state-of-the-art white-box
attacks. Our results show that inexpensive attacks are often as effective as
expensive meta-classifier attacks, and that there are surprising asymmetries in
the effectiveness of attacks. Code is available at
https://github.com/iamgroot42/FormEstDistRisks |
---|---|
DOI: | 10.48550/arxiv.2109.06024 |