CheXReport: A transformer-based architecture to generate chest X-ray reports suggestions

• Published in: Expert systems with applications Vol. 255; p. 124644
• Main Authors: Zeiser, Felipe André, da Costa, Cristiano André, de Oliveira Ramos, Gabriel, Maier, Andreas, da Rosa Righi, Rodrigo
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.12.2024
• Subjects: Radiology reports generation; Swin transformer; Transformer; X-ray; Radiology reports generation; Swin transformer; Transformer; X-ray
• ISSN: 0957-4174
• DOI: 10.1016/j.eswa.2024.124644

Chest X-ray analysis plays a fundamental role in modern medicine for screening, diagnosing, and defining treatment strategies. This importance exposes radiologists to high workloads and demands for detecting increasingly specific findings. The task of assisting the chest x-ray imaging process is clinically relevant. However, current literature still captures relevant visual information and correlates it with the reports’ findings. In this way, the proposal for automatic suggestions for X-ray reports can assist in the X-ray analysis processes in clinical routine. This paper presents CheXReport model, designed to generate chest X-ray reports by leveraging a fully transformer-based encoder–decoder framework. Unlike traditional approaches, our model uses Swin Transformer blocks in both the encoder and decoder, improving the extraction and integration of visual and textual features from chest X-ray images. We evaluate the CheXReport on the publicly available MIMIC-CXR dataset comprising 377,110 images and corresponding free-text reports. Specifically, CheXReport achieves state-of-the-art performance on the MIMIC-CXR dataset, outperforming other leading models on BLEU-4 and ROUGE metrics. Our qualitative and quantitative analyses highlight the effectiveness of the fully transformer-based architecture in generating detailed, accurate, and contextually relevant radiology reports. [Display omitted] •CheXReport leverages a fully transformer-based encoder–decoder architecture.•Enhance chest X-ray report generation on the MIMIC-CXR dataset.•CheXReport can correlate the images’ global context with the report to suggest radiological findings.