Deep Unlearning via Randomized Conditionally Independent Hessians

Recent legislation has led to interest in machine unlearning, i. e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For...

Full description

Saved in:
Bibliographic Details
Published inProceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) Vol. 2022; pp. 10412 - 10421
Main Authors Mehta, Ronak, Pal, Sourav, Singh, Vikas, Ravi, Sathya N.
Format Conference Proceeding Journal Article
LanguageEnglish
Published United States IEEE 01.06.2022
Subjects
accountability
Computational modeling
Face recognition
fairness
Law
Legislation
Predictive models
privacy and ethics in vision; Machine learning; Optimization methods; Statistical methods
Semantics
Training
Transparency
Online AccessGet full text

Cover

More Information
Summary:Recent legislation has led to interest in machine unlearning, i. e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person reidentification and NLP models that may require unlearning samples identified for exclusion. Code is available at https://github.com/vsingh-group/LCODEC-deep-unlearning
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Joint First Authors.
ISSN:1063-6919
1063-6919
2575-7075
DOI:10.1109/CVPR52688.2022.01017