COLIEE 2020: Methods for Legal Document Retrieval and Entailment

• Published in: New Frontiers in Artificial Intelligence Vol. 12758; pp. 196 - 210
• Main Authors: Rabelo, Juliano, Kim, Mi-Young, Goebel, Randy, Yoshioka, Masaharu, Kano, Yoshinobu, Satoh, Ken
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Classification; Information retrieval; Legal documents processing; Question answering; Textual entailment
• ISBN: 3030799417; 9783030799410
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-79942-7_13

We present a summary of the 7th Competition on Legal Information Extraction and Entailment. The competition consists of four tasks on case law and statute law. The case law component includes an information retrieval task (Task 1), and the confirmation of an entailment relation between an existing case and an unseen case (Task 2). The statute law component includes an information retrieval task (Task 3) and an entailment/question answering task (Task 4). Participation was open to any group based on any approach. Ten different teams participated in the case law competition tasks, most of them in more than one task. We received results from 9 teams for Task 1 (22 runs) and 8 teams for Task 2 (22 runs). On the statute law task, there were 14 different teams participating, most in more than one task. Eleven teams submitted a total of 28 runs for Task 3, and 13 teams submitted a total of 30 runs for Task 4. We summarize the approaches, our official evaluation, and analysis on our data and submission results.