Are generative AI text annotations systematically biased?

TL;DR

This study examines whether generative large language models systematically exhibit bias in text annotation tasks, revealing that while they perform adequately in accuracy, they differ from manual annotations and show overlapping biases.

Contribution

It provides a comprehensive analysis of bias in GLLM annotations across multiple models and prompts, highlighting systematic biases and limitations of F1 scores in capturing bias.

Findings

GLLMs perform adequately in F1 scores

GLLMs differ from manual annotations in prevalence

GLLMs exhibit systematic bias and overlap more with each other than with manual annotations

Abstract

This paper investigates bias in GLLM annotations by conceptually replicating manual annotations of Boukes (2024). Using various GLLMs (Llama3.1:8b, Llama3.3:70b, GPT4o, Qwen2.5:72b) in combination with five different prompts for five concepts (political content, interactivity, rationality, incivility, and ideology). We find GLLMs perform adequate in terms of F1 scores, but differ from manual annotations in terms of prevalence, yield substantively different downstream results, and display systematic bias in that they overlap more with each other than with manual annotations. Differences in F1 scores fail to account for the degree of bias.