Method and system for evaluating labeling quality - JIN JIAN; MA JIE; GUO PENG; DENG XIAONING; LIN WENCONG (2024)
The invention relates to a method and system for evaluating annotation quality, and belongs to the technical field of annotation corpora, and the method comprises the steps: obtaining sample data from a to-be-evaluated annotation corpus, and obtaining to-be-evaluated sample data; based on a unified labeling specification, labeling the original data of the to-be-evaluated sample data before labeling to obtain comparison sample data; calculating a consistency value of the to-be-evaluated sample data and the comparison sample data; and evaluating the annotation quality of the to-be-evaluated annotation corpus according to the consistency value. According to the method, the to-be-evaluated sample data and the comparison sample data are obtained through the unified labeling specification, the consistency value of the to-be-evaluated sample data and the comparison sample data is judged, and then the labeling quality of the to-be-evaluated sample data is evaluated. 本发明涉及一种评估标注质量的方法及系统，属于标注语料库技术领域，所述方法包括：从待评估标注语料库中获取