Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models

• Main Authors: Safavi-Naini, Seyed Amir Ahmad, Ali, Shuhaib, Shahab, Omer, Shahhoseini, Zahra, Savage, Thomas, Rafiee, Sara, Samaan, Jamil S, Shabeeb, Reem Al, Ladak, Farah, Yang, Jamie O, Echavarria, Juan, Babar, Sumbal, Shaukat, Aasma, Margolis, Samuel, Tatonetti, Nicholas P, Nadkarni, Girish, Kurdi, Bara El, Soroush, Ali
• Format: Journal Article
• Language: English
• Published: 25.08.2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language

Background and Aims: This study evaluates the medical reasoning performance of large language models (LLMs) and vision language models (VLMs) in gastroenterology. Methods: We used 300 gastroenterology board exam-style multiple-choice questions, 138 of which contain images to systematically assess th...