A dataset of clinically generated visual questions and answers about radiology images

• Published in: Scientific data Vol. 5; no. 1; pp. 180251 - 10
• Main Authors: Lau, Jason J., Gayen, Soumya, Ben Abacha, Asma, Demner-Fushman, Dina
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 20.11.2018; Nature Publishing Group
• Subjects: 631/114/2164; 692/700/1421/1770; Algorithms; Artificial intelligence; Cancer; Data Analysis; Data collection; Data Descriptor; Data Mining; Datasets; Decision making; Decision support systems; Humanities and Social Sciences; Humans; Image processing; Image Processing, Computer-Assisted - methods; Learning algorithms; Machine Learning; multidisciplinary; Radiography - methods; Radiology Information Systems - classification; Radiology Information Systems - standards; Science
• ISSN: 2052-4463; 2052-4463
• DOI: 10.1038/sdata.2018.251

Radiology images are an essential part of clinical decision making and population screening, e.g., for cancer. Automated systems could help clinicians cope with large amounts of images by answering questions about the image contents. An emerging area of artificial intelligence, Visual Question Answering (VQA) in the medical domain explores approaches to this form of clinical decision support. Success of such machine learning tools hinges on availability and design of collections composed of medical images augmented with question-answer pairs directed at the content of the image. We introduce VQA-RAD, the first manually constructed dataset where clinicians asked naturally occurring questions about radiology images and provided reference answers. Manual categorization of images and questions provides insight into clinically relevant tasks and the natural language to phrase them. Evaluating with well-known algorithms, we demonstrate the rich quality of this dataset over other automatically constructed ones. We propose VQA-RAD to encourage the community to design VQA tools with the goals of improving patient care. Design Type(s) image creation and editing objective • anatomical image analysis objective Measurement Type(s) image analysis Technology Type(s) visual observation method Factor Type(s) question type • answer type Sample Characteristic(s) Homo sapiens • head • chest • abdomen Machine-accessible metadata file describing the reported data (ISA-Tab format)