MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging

• Published in: arXiv.org
• Main Authors: Wang, Hongru, Wang, Zezhong, Wai Chung Kwan, Wong, Kam-Fai
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 11.09.2023
• Subjects: Deep learning; Marking; Optimization; Teaching methods
• ISSN: 2331-8422

Meta-learning is widely used for few-shot slot tagging in task of few-shot learning. The performance of existing methods is, however, seriously affected by \textit{sample forgetting issue}, where the model forgets the historically learned meta-training tasks while solely relying on support sets when adapting to new tasks. To overcome this predicament, we propose the \textbf{M}emory-based \textbf{C}ontrastive \textbf{M}eta-\textbf{L}earning (aka, MCML) method, including \textit{learn-from-the-memory} and \textit{adaption-from-the-memory} modules, which bridge the distribution gap between training episodes and between training and testing respectively. Specifically, the former uses an explicit memory bank to keep track of the label representations of previously trained episodes, with a contrastive constraint between the label representations in the current episode with the historical ones stored in the memory. In addition, the \emph{adaption-from-memory} mechanism is introduced to learn more accurate and robust representations based on the shift between the same labels embedded in the testing episodes and memory. Experimental results show that the MCML outperforms several state-of-the-art methods on both SNIPS and NER datasets and demonstrates strong scalability with consistent improvement when the number of shots gets greater.