MIRACL : A Multilingual Retrieval Dataset Covering 18 Diverse Languages

• Published in: Transactions of the Association for Computational Linguistics Vol. 11; pp. 1114 - 1131
• Main Authors: Zhang, Xinyu, Thakur, Nandan, Ogundepo, Odunayo, Kamalloo, Ehsan, Alfonso-Hermelo, David, Li, Xiaoguang, Liu, Qun, Rezagholizadeh, Mehdi, Lin, Jimmy
• Format: Journal Article
• Language: English
• Published: One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA MIT Press 01.09.2023; MIT Press Journals, The; The MIT Press
• Subjects: Annotations; Computational linguistics; Control data (computers); Datasets; Language diversity; Languages; Multilingualism; Native speakers; Queries; Retrieval
• ISSN: 2307-387X; 2307-387X
• DOI: 10.1162/tacl_a_00595

MIRACL is a multilingual dataset for retrieval across 18 languages that collectively encompass over three billion native speakers around the world. This resource is designed to support monolingual retrieval tasks, where the queries and the corpora are in the same language. In total, we have gathered over 726k high-quality relevance judgments for 78k queries over Wikipedia in these languages, where all annotations have been performed by native speakers hired by our team. MIRACL covers languages that are both typologically close as well as distant from 10 language families and 13 sub-families, associated with varying amounts of publicly available resources. Extensive automatic heuristic verification and manual assessments were performed during the annotation process to control data quality. In total, MIRACL represents an investment of around five person-years of human annotator effort. Our goal is to spur research on improving retrieval across a continuum of languages, thus enhancing information access capabilities for diverse populations around the world, particularly those that have traditionally been underserved. MIRACL is available at .