# High risk of political bias in black box emotion inference models

## TL;DR

This paper shows that emotion inference models can have political bias, affecting how they interpret text and suggesting caution in their use for research.

## Contribution

The study reveals political bias in a sentiment analysis model and proposes lexicon-based systems as a neutral alternative.

## Key findings

- Political affiliations influence valence predictions in a Polish sentiment analysis model.
- Removing texts mentioning politicians reduced bias but did not eliminate it entirely.
- Human annotations propagate political biases into model predictions.

## Abstract

This paper investigates the presence of political bias in emotion inference models used for sentiment analysis (SA). Machine learning models often reflect biases in their training data, impacting the validity of their outcomes. While previous research has highlighted gender and race biases, our study focuses on political bias—an underexplored, pervasive issue that can skew the interpretation of text data across many studies. We audit a Polish sentiment analysis model developed in our lab for bias. By analyzing valence predictions for names and sentences involving Polish politicians, we uncovered systematic differences influenced by political affiliations. Our findings suggest that annotations by human raters propagate political biases into the model’s predictions. To prove it, we pruned the training dataset of texts mentioning these politicians and observed a reduction in bias, though not its complete elimination. Given the significant implications of political bias in SA, our study emphasizes caution in employing these models for social science research. We recommend a critical examination of SA results and propose using lexicon-based systems as an ideologically neutral alternative. This paper underscores the necessity for ongoing scrutiny and methodological adjustments to ensure the reliability of the use of machine learning in academic and applied contexts.

The online version contains supplementary material available at 10.1038/s41598-025-86766-6.