TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization

• Main Authors: Tang, Liyan, Shalyminov, Igor, Wong, Amy Wing-mei, Burnsky, Jon, Vincent, Jake W, Yang, Yu'an, Singh, Siffi, Feng, Song, Song, Hwanjun, Su, Hang, Sun, Lijia, Zhang, Yi, Mansour, Saab, McKeown, Kathleen
• Format: Journal Article
• Language: English
• Published: 20.02.2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language

Single document news summarization has seen substantial progress on faithfulness in recent years, driven by research on the evaluation of factual consistency, or hallucinations. We ask whether these advances carry over to other text summarization domains. We propose a new evaluation benchmark on top...