Studies Evaluating Artificial Intelligence (AI) Large Language Models Ability to Respond to Questions Are Repetitive and Out-of-Date: AI Must Now Be Applied to Improving Clinical Practice and Patient Care

• Published in: Arthroscopy
• Main Author: Oeding, Jacob F.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 24.10.2024
• ISSN: 0749-8063; 1526-3231; 1526-3231
• DOI: 10.1016/j.arthro.2024.10.020

While artificial intelligence (AI) technologies like ChatGPT have demonstrated very real and powerful capabilities to date, this does not mean that research studying these technologies is immune from “shiny object” syndrome, a psychological phenomenon where we tend to focus on new and fashionable ideas only to be distracted from those that truly matter. In parallel with the increased publicity that AI has received since the release of large language models (LLMs) like ChatGPT has been an explosion in the number of studies evaluating LLMs’ ability to answer hypothetical questions from patients on a variety of conditions. Nevertheless, these studies tend to leave us with the same conclusion: LLMs are generally capable of providing reliable and relevant responses to patient questions but are not without limitations. Given the abundance of studies demonstrating similar outcomes regardless of whether the LLMs are asked to respond to a patient’s questions about their diabetes or about their shoulder dislocation, I’m afraid we are at risk of making AI more of a “shiny object” than a tool that can be used to change clinical practice and improve patient care. Specifically, we may be getting to a point where a “publish or perish” mindset has promoted studies with repetitive methodologies that only confirm well-established theories around the capabilities and limitations of AI and has created a distraction from new use-cases and more meaningful applications for patient care. We are now at a crossroads where we can either remain stuck in the past, repeating old studies’ methodologies on a different procedure or injury, or progress by expanding the number and impact of applications these tools have in orthopaedic surgery. The capabilities of AI will continue to increase at rapid pace, but it will be up to those with intricate knowledge of orthopaedics and patient care to keep up.