Can Artificial Intelligence Improve the Readability of Patient Education Materials on Aortic Stenosis? A Pilot Study

• Published in: Cardiology and Therapy Vol. 13; no. 1; pp. 137 - 147
• Main Authors: Rouhi, Armaun D., Ghanem, Yazid K., Yolchieva, Laman, Saleh, Zena, Joshi, Hansa, Moccia, Matthew C., Suarez-Pierre, Alejandro, Han, Jason J.
• Format: Journal Article
• Language: English
• Published: Cheshire Springer Healthcare 01.03.2024; Springer; Adis, Springer Healthcare
• Subjects: Analysis; Aortic stenosis; Aortic valve stenosis; Artificial intelligence; Cardiac patients; Cardiology; Cardiovascular diseases; Care and treatment; ChatGPT; Health literacy; Heart valve disease; Internal Medicine; Literacy; Medicine; Medicine & Public Health; Original Research; Patient education; Patient education material; Readability; Technology application; United States; Aortic stenosis; Health literacy; Heart valve disease; Readability; Large language models; ChatGPT; Patient education material; Chatbots; Artificial intelligence
• ISSN: 2193-8261; 2193-6544
• DOI: 10.1007/s40119-023-00347-0

Introduction The advent of generative artificial intelligence (AI) dialogue platforms and large language models (LLMs) may help facilitate ongoing efforts to improve health literacy. Additionally, recent studies have highlighted inadequate health literacy among patients with cardiac disease. The aim of the present study was to ascertain whether two freely available generative AI dialogue platforms could rewrite online aortic stenosis (AS) patient education materials (PEMs) to meet recommended reading skill levels for the public. Methods Online PEMs were gathered from a professional cardiothoracic surgical society and academic institutions in the USA. PEMs were then inputted into two AI-powered LLMs, ChatGPT-3.5 and Bard, with the prompt “translate to 5th-grade reading level”. Readability of PEMs before and after AI conversion was measured using the validated Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook Index (SMOGI), and Gunning-Fog Index (GFI) scores. Results Overall, 21 PEMs on AS were gathered. Original readability measures indicated difficult readability at the 10th–12th grade reading level. ChatGPT-3.5 successfully improved readability across all four measures ( p  < 0.001) to the approximately 6th–7th grade reading level. Bard successfully improved readability across all measures ( p  < 0.001) except for SMOGI ( p  = 0.729) to the approximately 8th–9th grade level. Neither platform generated PEMs written below the recommended 6th-grade reading level. ChatGPT-3.5 demonstrated significantly more favorable post-conversion readability scores, percentage change in readability scores, and conversion time compared to Bard (all p  < 0.001). Conclusion AI dialogue platforms can enhance the readability of PEMs for patients with AS but may not fully meet recommended reading skill levels, highlighting potential tools to help strengthen cardiac health literacy in the future.