CJRC: A Reliable Human-Annotated Benchmark DataSet for Chinese Judicial Reading Comprehension

• Published in: Chinese Computational Linguistics pp. 439 - 451
• Main Authors: Duan, Xingyi, Wang, Baoxin, Wang, Ziyue, Ma, Wentao, Cui, Yiming, Wu, Dayong, Wang, Shijin, Liu, Ting, Huo, Tianxiang, Hu, Zhen, Wang, Heng, Liu, Zhiyuan
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing 2019
• Series: Lecture Notes in Computer Science
• ISBN: 3030323803; 9783030323806
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-32381-3_36

We present a Chinese judicial reading comprehension (CJRC) dataset which contains approximately 10K documents and almost 50K questions with answers. The documents come from judgment documents and the questions are annotated by law experts. The CJRC dataset can help researchers extract elements by reading comprehension technology. Element extraction is an important task in the legal field. However, it is difficult to predefine the element types completely due to the diversity of document types and causes of action. By contrast, machine reading comprehension technology can quickly extract elements by answering various questions from the long document. We build two strong baseline models based on BERT and BiDAF. The experimental results show that there is enough space for improvement compared to human annotators.