Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data

When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a tr...

Full description

Saved in:
Bibliographic Details
Published inScientific reports Vol. 13; no. 1; p. 10666
Main Authors Khader, Firas, Kather, Jakob Nikolas, Müller-Franzes, Gustav, Wang, Tianci, Han, Tianyu, Tayebi Arasteh, Soroosh, Hamesch, Karim, Bressem, Keno, Haarburger, Christoph, Stegmaier, Johannes, Kuhl, Christiane, Nebelung, Sven, Truhn, Daniel
Format Journal Article
LanguageEnglish
Published London Nature Publishing Group UK 01.07.2023
Nature Publishing Group
Nature Portfolio
Subjects
639/705/117
692/700/1421
692/700/1421/1770
Area Under Curve
Critical Care
Diagnostic Imaging
Electric Power Supplies
Electronic health records
Hospitals
Humanities and Social Sciences
Humans
Intensive care
Machine learning
Medical prognosis
multidisciplinary
Neural networks
Patients
Performance evaluation
Physicians
Radiography
Retrospective Studies
Science
Science (multidisciplinary)
Sensory integration
Survival
Online AccessGet full text

Cover

More Information
Summary:When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a transformer-based neural network as a novel AI architecture that integrates multimodal patient data, i.e., imaging data (chest radiographs) and non-imaging data (clinical data). We evaluate the performance of our model in a retrospective study with 6,125 patients in intensive care. We show that the combined model (area under the receiver operating characteristic curve [AUROC] of 0.863) is superior to the radiographs-only model (AUROC = 0.811, p < 0.001) and the clinical data-only model (AUROC = 0.785, p < 0.001) when tasked with predicting in-hospital survival per patient. Furthermore, we demonstrate that our proposed model is robust in cases where not all (clinical) data points are available.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-023-37835-1