Soft-Label Integration for Robust Toxicity Classification

TL;DR

This paper presents a novel bi-level optimization framework that combines crowdsourced soft-labels with GroupDRO to improve the robustness and accuracy of toxicity classifiers, especially against out-of-distribution data.

Contribution

It introduces a new bi-level optimization method integrating crowdsourced annotations with soft-labeling and GroupDRO for robust toxicity classification.

Findings

Outperforms baseline methods in accuracy metrics

Enhances robustness against out-of-distribution data

Theoretically proven convergence of the optimization algorithm

Abstract

Toxicity classification in textual content remains a significant problem. Data with labels from a single annotator fall short of capturing the diversity of human perspectives. Therefore, there is a growing need to incorporate crowdsourced annotations for training an effective toxicity classifier. Additionally, the standard approach to training a classifier using empirical risk minimization (ERM) may fail to address the potential shifts between the training set and testing set due to exploiting spurious correlations. This work introduces a novel bi-level optimization framework that integrates crowdsourced annotations with the soft-labeling technique and optimizes the soft-label weights by Group Distributionally Robust Optimization (GroupDRO) to enhance the robustness against out-of-distribution (OOD) risk. We theoretically prove the convergence of our bi-level optimization algorithm. Experimental results demonstrate that our approach outperforms existing baseline methods in terms of both average and worst-group accuracy, confirming its effectiveness in leveraging crowdsourced annotations to achieve more effective and robust toxicity classification.