Mitigating Unintended Memorization in Language Models via Alternating Teaching

• Published in: arXiv.org
• Main Authors: Liu, Zhe, Zhang, Xuedong, Peng, Fuchun
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 13.10.2022
• Subjects: Privacy; Teachers; Training

Recent research has shown that language models have a tendency to memorize rare or unique sequences in the training corpora which can thus leak sensitive attributes of user data. We employ a teacher-student framework and propose a novel approach called alternating teaching to mitigate unintended mem...