A worker clustering-based approach of label aggregation under the belief function theory

• Published in: Applied intelligence (Dordrecht, Netherlands) Vol. 49; no. 1; pp. 53 - 62
• Main Authors: Abassi, Lina, Boukhris, Imen
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.01.2019; Springer Nature B.V
• Subjects: Agglomeration; Artificial Intelligence; Clustering; Computer Science; Labels; Machines; Manufacturing; Mechanical Engineering; Processes; Crowdsourcing; Label aggregation; Worker clustering; Belief function theory; Gold data
• ISSN: 0924-669X; 1573-7497
• DOI: 10.1007/s10489-018-1209-z

Crowdsourcing platforms have been attracting a wide attention in the field of artificial intelligence in recent years, providing a cheap and reachable human-powered resource to gather massive labeled data. These data are used to effectively build supervised learning models for academic research puposes. However, despite the attractiveness of these systems, the major concern has always been the quality of the collected labels. In fact, a wide range of workers contributes in labeling data leading to be in possession of potentially noisy and imperfect labels. Therefore in this paper, we propose a new label aggregation technique that allows to determine workers qualities via a clustering process and then represent and combine their labels to estimate the final one under the belief function theory. This latter is notorious for its strength and flexibility when dealing with imperfect information. Experimental results demonstrate that our proposed method outperforms the related work baseline and improves results quality.