Toxicity Inspector: A Framework to Evaluate Ground Truth in Toxicity Detection Through Feedback

Toxic language is difficult to define, as it is not monolithic and has many variations in perceptions of toxicity. This challenge of detecting toxic language is increased by the highly contextual and subjectivity of its interpretation, which can degrade the reliability of datasets and negatively aff...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Althunayan, Huriyyah, Bahlas, Rahaf, Alharbi, Manar, Alsuwailem, Lena, Aldayel, Abeer, ALahmadi, Rehab
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 11.05.2023
Subjects
Datasets
Feedback
Iterative methods
Reliability
Toxicity
Online AccessGet full text

Cover

More Information
Summary:Toxic language is difficult to define, as it is not monolithic and has many variations in perceptions of toxicity. This challenge of detecting toxic language is increased by the highly contextual and subjectivity of its interpretation, which can degrade the reliability of datasets and negatively affect detection model performance. To fill this void, this paper introduces a toxicity inspector framework that incorporates a human-in-the-loop pipeline with the aim of enhancing the reliability of toxicity benchmark datasets by centering the evaluator's values through an iterative feedback cycle. The centerpiece of this framework is the iterative feedback process, which is guided by two metric types (hard and soft) that provide evaluators and dataset creators with insightful examination to balance the tradeoff between performance gains and toxicity avoidance.
ISSN:2331-8422