A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Prediction
Mobile devices, especially smartphones, can support rich functions and have developed into indispensable tools in daily life. With the rise of generative AI services, smartphones can potentially transform into personalized assistants, anticipating user needs and scheduling services accordingly. Pred...
Saved in:
Main Authors | , , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
19.08.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Mobile devices, especially smartphones, can support rich functions and have
developed into indispensable tools in daily life. With the rise of generative
AI services, smartphones can potentially transform into personalized
assistants, anticipating user needs and scheduling services accordingly.
Predicting user intents on smartphones, and reflecting anticipated activities
based on past interactions and context, remains a pivotal step towards this
vision. Existing research predominantly focuses on specific domains, neglecting
the challenge of modeling diverse event sequences across dynamic contexts.
Leveraging pre-trained language models (PLMs) offers a promising avenue, yet
adapting PLMs to on-device user intent prediction presents significant
challenges. To address these challenges, we propose PITuning, a
Population-to-Individual Tuning framework. PITuning enhances common pattern
extraction through dynamic event-to-intent transition modeling and addresses
long-tailed preferences via adaptive unlearning strategies. Experimental
results on real-world datasets demonstrate PITuning's superior intent
prediction performance, highlighting its ability to capture long-tailed
preferences and its practicality for on-device prediction scenarios. |
---|---|
DOI: | 10.48550/arxiv.2408.09815 |