ChatGPT M.D.: Is there any room for generative AI in neurology?

• Published in: PloS one Vol. 19; no. 10; p. e0310028
• Main Authors: Nógrádi, Bernát, Polgár, Tamás Ferenc, Meszlényi, Valéria, Kádár, Zalán, Hertelendy, Péter, Csáti, Anett, Szpisjak, László, Halmi, Dóra, Erdélyi-Furka, Barbara, Tóth, Máté, Molnár, Fanny, Tóth, Dávid, Bősze, Zsófia, Boda, Krisztina, Klivényi, Péter, Siklós, László, Patai, Roland
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 09.10.2024
• Subjects: Accuracy; Adult; Artificial Intelligence; Biology and Life Sciences; Chatbots; Computer and Information Sciences; Data augmentation; Diagnosis; Emergency medical care; Evaluation; Female; General Data Protection Regulation; Generative artificial intelligence; Humans; Ischemia; Language; Large language models; Male; Medical history; Medical research; Medicine and Health Sciences; Medicine, Experimental; Middle Aged; Nervous System Diseases - diagnosis; Neurological diseases; Neurological disorders; Neurology; Neurology - methods; Patient assessment; People and Places; Physicians; Sensitivity analysis; Social Sciences; Stroke; Synthetic data; Technology application; Hungary; United States > US
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0310028

ChatGPT, a general artificial intelligence, has been recognized as a powerful tool in scientific writing and programming but its use as a medical tool is largely overlooked. The general accessibility, rapid response time and comprehensive training database might enable ChatGPT to serve as a diagnostic augmentation tool in certain clinical settings. The diagnostic process in neurology is often challenging and complex. In certain time-sensitive scenarios, rapid evaluation and diagnostic decisions are needed, while in other cases clinicians are faced with rare disorders and atypical disease manifestations. Due to these factors, the diagnostic accuracy in neurology is often suboptimal. Here we evaluated whether ChatGPT can be utilized as a valuable and innovative diagnostic augmentation tool in various neurological settings. We used synthetic data generated by neurological experts to represent descriptive anamneses of patients with known neurology-related diseases, then the probability for an appropriate diagnosis made by ChatGPT was measured. To give clarity to the accuracy of the AI-determined diagnosis, all cases have been cross-validated by other experts and general medical doctors as well. We found that ChatGPT-determined diagnostic accuracy (ranging from 68.5% ± 3.28% to 83.83% ± 2.73%) can reach the accuracy of other experts (81.66% ± 2.02%), furthermore, it surpasses the probability of an appropriate diagnosis if the examiner is a general medical doctor (57.15% ± 2.64%). Our results showcase the efficacy of general artificial intelligence like ChatGPT as a diagnostic augmentation tool in medicine. In the future, AI-based supporting tools might be useful amendments in medical practice and help to improve the diagnostic process in neurology.