RecPrompt: A Self-tuning Prompting Framework for News Recommendation Using Large Language Models

• Published in: arXiv.org
• Main Authors: Liu, Dairui, Yang, Boming, Du, Honghui, Greene, Derek, Hurley, Neil, Lawlor, Aonghus, Dong, Ruihai, Li, Irene
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 22.10.2024
• Subjects: Computer Science - Information Retrieval; Iterative methods; Large language models; News; Prompt engineering; Recommender systems; Semantics
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.2312.10463

News recommendations heavily rely on Natural Language Processing (NLP) methods to analyze, understand, and categorize content, enabling personalized suggestions based on user interests and reading behaviors. Large Language Models (LLMs) like GPT-4 have shown promising performance in understanding natural language. However, the extent of their applicability to news recommendation systems remains to be validated. This paper introduces RecPrompt, the first self-tuning prompting framework for news recommendation, leveraging the capabilities of LLMs to perform complex news recommendation tasks. This framework incorporates a news recommender and a prompt optimizer that applies an iterative bootstrapping process to enhance recommendations through automatic prompt engineering. Extensive experimental results with 400 users show that RecPrompt can achieve an improvement of 3.36% in AUC, 10.49% in MRR, 9.64% in nDCG@5, and 6.20% in nDCG@10 compared to deep neural models. Additionally, we introduce TopicScore, a novel metric to assess explainability by evaluating LLM's ability to summarize topics of interest for users. The results show LLM's effectiveness in accurately identifying topics of interest and delivering comprehensive topic-based explanations.