Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels - Jouaiti, Melanie; Dautenhahn, Kerstin (Speech and Computer, 2022)
Manually labelled datasets inherently contain errors or uncertain/imprecise labelling as sometimes experts cannot agree or are not sure. This issue is even more prominent in multi-label datasets as some labels may be missing. However, discarding samples with high uncertainty may lead to the loss of valuable data. In this paper, we study two datasets where the uncertainty is explicit in the expert annotations. We give an overview of the different approaches available to deal with uncertainty and evaluate them on two dysfluency datasets. Our results show that adopting methods that embrace uncertainty leads to better results than using only labels with high certainty and performs better than current state of the art results.