Method and system for evaluating labeling quality

• Main Authors: JIN JIAN, MA JIE, GUO PENG, DENG XIAONING, LIN WENCONG
• Format: Patent
• Language: Chinese; English
• Published: 24.05.2024
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention relates to a method and system for evaluating annotation quality, and belongs to the technical field of annotation corpora, and the method comprises the steps: obtaining sample data from a to-be-evaluated annotation corpus, and obtaining to-be-evaluated sample data; based on a unified labeling specification, labeling the original data of the to-be-evaluated sample data before labeling to obtain comparison sample data; calculating a consistency value of the to-be-evaluated sample data and the comparison sample data; and evaluating the annotation quality of the to-be-evaluated annotation corpus according to the consistency value. According to the method, the to-be-evaluated sample data and the comparison sample data are obtained through the unified labeling specification, the consistency value of the to-be-evaluated sample data and the comparison sample data is judged, and then the labeling quality of the to-be-evaluated sample data is evaluated. 本发明涉及一种评估标注质量的方法及系统，属于标注语料库技术领域，所述方法包括：从待评估标注语料库中获取