You believe your LLM is not delusional? Think again! a study of LLM hallucination on foundation models under perturbation

• Published in: Discover data Vol. 3; no. 1; pp. 1 - 9
• Main Authors: Saha, Anirban, Gupta, Binay, Chatterjee, Anirban, Banerjee, Kunal
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 29.05.2025; Springer
• Subjects: Big Data/Analytics; Computer Science; Data Engineering; Google Gemini; Large language model (LLM); LLM hallucination; OpenAI GPT; Perturbation; OpenAI GPT; Perturbation; Google Gemini; Large language model (LLM); LLM hallucination
• ISSN: 2731-6955; 2731-6955
• DOI: 10.1007/s44248-025-00041-7

Large Language Model (LLM) has recently become almost a household term because of its wide range of applications and immense popularity. However, hallucination in LLMs is a critical issue as it affects the quality of an LLM’s response, reduces user trust and leads to the spread of misinformation. Detecting hallucination in the presence of the context or a golden response is relatively easier but it becomes considerably more challenging when both of these are absent, which is typically the case post deployment of an LLM. In this study, we present a framework that relies on query perturbation and consistency score calculation between the responses generated against the original query and the perturbed query to identify the potential hallucination scenarios. This framework has no dependency on the availability of the context or the ground truth. In this study, we focus on the popular foundation models because majority of the LLM applications leverage these specific models since training an LLM from scratch or even finetuning LLMs may require a lot of capital investment. Moreover, we specifically investigate LLM hallucinations under different levels of perturbation: character-level, word-level and sentence-level — robustness towards these perturbations indicates that an LLM has a good understanding of a concept, and thus is less susceptible to hallucinations – this, in turn, should help in the LLM’s user adoption. Our study shows that GPT-4 hallucinates the least when faced with perturbations; in contrast, other LLMs start hallucinating even with minor typos.