An Evidential Semi-supervised Label Aggregation Approach

• Published in: Knowledge Science, Engineering and Management pp. 676 - 686
• Main Authors: Abassi, Lina, Boukhris, Imen
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Answer aggregation; Crowdsourcing; Evidence theory; Gold labels
• ISBN: 9783030295509; 3030295508
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-29551-6_60

Crowdsourcing is a powerful concept that typically takes advantage of human intelligence to deal with problems in many fields most importantly in machine learning. Indeed, it enables to collect training labels in a fast and cheap way for supervised algorithms. The only major challenge is that the quality of the contributions is not always guaranteed because of the expertise heterogeneity of the participants. One of the basic strategies to overcome this problem is to assign each task to multiple workers and then combine their answers in order to obtain a single reliable one. This paper provides a new iterative approach that aggregates imperfect labels using the supervision of few gold labels under the evidence theory. Besides of inferring the consensus answers, the workers’ accuracies and the questions difficulties are as well estimated. A comparative evaluation on synthetic and real datasets confirms the effectiveness of our semi-supervised approach over the baselines.