Multimodal medical information retrieval with unsupervised rank fusion

• Published in: Computerized medical imaging and graphics Vol. 39; pp. 35 - 45
• Main Authors: Mourão, André, Martins, Flávio, Magalhães, João
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.01.2015
• Subjects: Algorithms; Artificial Intelligence; Assisted query formulation; Data fusion; Data integration; Documentation - methods; Health care; Information retrieval; Information Storage and Retrieval - methods; Internal Medicine; Medical; Medical search; Medical Subject Headings; Multimodal Imaging - methods; Multimodal retrieval; Natural Language Processing; Other; Pattern Recognition, Automated - methods; PubMed; Radiology Information Systems - organization & administration; Retrieval; Search interfaces; Searching; Subtraction Technique; Terminology as Topic; Thesauri; User-Computer Interface; Vocabulary, Controlled; Medical search; Search interfaces; Data fusion; Assisted query formulation; Multimodal retrieval
• ISSN: 0895-6111; 1879-0771; 1879-0771
• DOI: 10.1016/j.compmedimag.2014.05.006

•We propose a medical retrieval system supporting text and image queries.•It can retrieve either relevant PubMed articles or images from those articles.•It supports automatic query term expansion from the MeSH thesaurus.•Novel fusion algorithm, ISR, improves results from existing rank fusion algorithms.•We got the best result on multimodal case-based retrieval in 2013 ImageCLEFMedical Modern medical information retrieval systems are paramount to manage the insurmountable quantities of clinical data. These systems empower health care experts in the diagnosis of patients and play an important role in the clinical decision process. However, the ever-growing heterogeneous information generated in medical environments poses several challenges for retrieval systems. We propose a medical information retrieval system with support for multimodal medical case-based retrieval. The system supports medical information discovery by providing multimodal search, through a novel data fusion algorithm, and term suggestions from a medical thesaurus. Our search system compared favorably to other systems in 2013 ImageCLEFMedical.