Assessing the Reliability of LLMs Annotations in the Context of Demographic Bias and Model Explanation

TL;DR

This paper investigates the influence of demographic bias on annotations in NLP tasks, evaluates the reliability of generative AI as annotators, and emphasizes content-based explanations for fairer models.

Contribution

It quantifies demographic influence on annotations, assesses GenAI annotation reliability, and advocates content-focused explanations over demographic personas for fairness.

Findings

Demographic factors account for only 8% of annotation variance.

Guided GenAI models do not outperform baseline models in alignment with human judgments.

Content-specific tokens are primary drivers of model predictions, not demographic features.

Abstract

Understanding the sources of variability in annotations is crucial for developing fair NLP systems, especially for tasks like sexism detection where demographic bias is a concern. This study investigates the extent to which annotator demographic features influence labeling decisions compared to text content. Using a Generalized Linear Mixed Model, we quantify this inf luence, finding that while statistically present, demographic factors account for a minor fraction ( 8%) of the observed variance, with tweet content being the dominant factor. We then assess the reliability of Generative AI (GenAI) models as annotators, specifically evaluating if guiding them with demographic personas improves alignment with human judgments. Our results indicate that simplistic persona prompting often fails to enhance, and sometimes degrades, performance compared to baseline models. Furthermore, explainable AI (XAI) techniques reveal that model predictions rely heavily on content-specific tokens related to sexism, rather than correlates of demographic characteristics. We argue that focusing on content-driven explanations and robust annotation protocols offers a more reliable path towards fairness than potentially persona simulation.