How AI should be used in radiology: assessing ambiguity and completeness of intended use statements of commercial AI products

• Published in: Insights into imaging Vol. 15; no. 1; p. 51
• Main Authors: van Leeuwen, Kicky G., Hedderich, Dennis M., Harvey, Hugh, Schalekamp, Steven
• Format: Journal Article
• Language: English
• Published: Vienna Springer Vienna 16.02.2024; Springer Nature B.V; SpringerOpen
• Subjects: Ambiguity; Artificial intelligence; Body parts; Completeness; Device approval process; Diagnostic Radiology; Imaging; Internal Medicine; Interventional Radiology; Labels; Medical device industry; Medical device legislation; Medical device safety; Medical devices; Medical equipment; Medicine; Medicine & Public Health; Neuroradiology; Original Article; Radiology; Ultrasound; Radiology; Medical device legislation; Medical device safety; Artificial intelligence; Device approval process
• ISSN: 1869-4101; 1869-4101
• DOI: 10.1186/s13244-024-01616-9

Background Intended use statements (IUSs) are mandatory to obtain regulatory clearance for artificial intelligence (AI)-based medical devices in the European Union. In order to guide the safe use of AI-based medical devices, IUSs need to contain comprehensive and understandable information. This study analyzes the IUSs of CE-marked AI products listed on AIforRadiology.com for ambiguity and completeness. Methods We retrieved 157 IUSs of CE-marked AI products listed on AIforRadiology.com in September 2022. Duplicate products ( n  = 1), discontinued products ( n  = 3), and duplicate statements ( n  = 14) were excluded. The resulting IUSs were assessed for the presence of 6 items: medical indication, part of the body, patient population, user profile, use environment, and operating principle. Disclaimers, defined as contra-indications or warnings in the IUS, were identified and compared with claims. Results Of 139 AI products, the majority ( n  = 78) of IUSs mentioned 3 or less items. IUSs of only 7 products mentioned all 6 items. The intended body part ( n  = 115) and the operating principle ( n  = 116) were the most frequently mentioned components, while the intended use environment ( n  = 24) and intended patient population ( n  = 29) were mentioned less frequently. Fifty-six statements contained disclaimers that conflicted with the claims in 13 cases. Conclusion The majority of IUSs of CE-marked AI-based medical devices lack substantial information and, in few cases, contradict the claims of the product. Critical relevance statement To ensure correct usage and to avoid off-label use or foreseeable misuse of AI-based medical devices in radiology, manufacturers are encouraged to provide more comprehensive and less ambiguous intended use statements. Key points • Radiologists must know AI products’ intended use to avoid off-label use or misuse. • Ninety-five percent ( n  = 132/139) of the intended use statements analyzed were incomplete. • Nine percent ( n  = 13) of the intended use statements held disclaimers contradicting the claim of the AI product. • Manufacturers and regulatory bodies must ensure that intended use statements are comprehensive. Graphical Abstract