Classification performance and reproducibility of GPT-4 omni for information extraction from veterinary electronic health records

• Published in: arXiv.org
• Main Authors: Wulcan, Judit M, Jacques, Kevin L, Lee, Mary Ann, Kovacs, Samantha L, Dausend, Nicole, Prince, Lauren E, Wulcan, Jonatan, Marsilio, Sina, Keller, Stefan M
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 09.09.2024
• Subjects: Ambiguity; Classification; Electronic health records; Errors; Human performance; Information retrieval; Large language models; Performance evaluation; Reproducibility; Sensitivity; Temperature effects

Large language models (LLMs) can extract information from veterinary electronic health records (EHRs), but performance differences between models, the effect of temperature settings, and the influence of text ambiguity have not been previously evaluated. This study addresses these gaps by comparing...