Accurate and Data-Efficient Toxicity Prediction when Annotators Disagree

TL;DR

This paper introduces new methods for predicting individual annotator ratings in toxicity detection, especially when annotators disagree, by leveraging annotator-specific data and demographics, improving prediction accuracy.

Contribution

The paper presents three novel approaches for modeling individual annotator ratings, highlighting the effectiveness of embedding-based architectures and survey-derived demographics in subjective NLP tasks.

Findings

Embedding-based architecture outperforms other methods.

Demographics from survey data are nearly as effective as true demographics.

Integrating annotator history and demographics improves rating prediction accuracy.

Abstract

When annotators disagree, predicting the labels given by individual annotators can capture nuances overlooked by traditional label aggregation. We introduce three approaches to predicting individual annotator ratings on the toxicity of text by incorporating individual annotator-specific information: a neural collaborative filtering (NCF) approach, an in-context learning (ICL) approach, and an intermediate embedding-based architecture. We also study the utility of demographic information for rating prediction. NCF showed limited utility; however, integrating annotator history, demographics, and survey information permits both the embedding-based architecture and ICL to substantially improve prediction accuracy, with the embedding-based architecture outperforming the other methods. We also find that, if demographics are predicted from survey information, using these imputed demographics as features performs comparably to using true demographic data. This suggests that demographics may not provide substantial information for modeling ratings beyond what is captured in survey responses. Our findings raise considerations about the relative utility of different types of annotator information and provide new approaches for modeling annotators in subjective NLP tasks.