Toxicity Inspector: A Framework to Evaluate Ground Truth in Toxicity Detection Through Feedback
Huriyyah Althunayan, Rahaf Bahlas, Manar Alharbi, Lena Alsuwailem,, Abeer Aldayel, Rehab ALahmadi

TL;DR
This paper presents a toxicity inspector framework that uses human feedback to improve the reliability of toxicity datasets, addressing the challenges of subjectivity and contextuality in toxic language detection.
Contribution
It introduces a human-in-the-loop framework with iterative feedback and dual metrics to enhance dataset reliability and balance detection performance with toxicity avoidance.
Findings
Framework improves dataset reliability
Iterative feedback balances performance and toxicity
Dual metrics provide insightful evaluation
Abstract
Toxic language is difficult to define, as it is not monolithic and has many variations in perceptions of toxicity. This challenge of detecting toxic language is increased by the highly contextual and subjectivity of its interpretation, which can degrade the reliability of datasets and negatively affect detection model performance. To fill this void, this paper introduces a toxicity inspector framework that incorporates a human-in-the-loop pipeline with the aim of enhancing the reliability of toxicity benchmark datasets by centering the evaluator's values through an iterative feedback cycle. The centerpiece of this framework is the iterative feedback process, which is guided by two metric types (hard and soft) that provide evaluators and dataset creators with insightful examination to balance the tradeoff between performance gains and toxicity avoidance.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Machine Learning and Data Classification · Anomaly Detection Techniques and Applications
