Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization

• Published in: arXiv.org
• Main Authors: Pan, Kaihang, Li, Juncheng, Song, Hongye, Lin, Jun, Liu, Xiaozhong, Tang, Siliang
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 23.10.2023
• Subjects: Domains; Learning; Regularization; Regularization methods; Training

Prompt tuning is a parameter-efficient method, which learns soft prompts and conditions frozen language models to perform specific downstream tasks. Though effective, prompt tuning under few-shot settings on the one hand heavily relies on a good initialization of soft prompts. On the other hand, it...