Inflating Topic Relevance with Ideology: A Case Study of Political Ideology Bias in Social Topic Detection Models

TL;DR

This paper studies how political ideology biases in training data affect social topic detection models, revealing bias propagation issues and proposing a method to create ideology-invariant representations to improve accuracy.

Contribution

It demonstrates the impact of political bias on NLP models and introduces a novel approach to mitigate bias by learning invariant text representations.

Findings

Bias propagates in NLP models affecting retrieval accuracy

Large models are highly susceptible to input biases

Proposed invariant representation reduces ideology bias

Abstract

We investigate the impact of political ideology biases in training data. Through a set of comparison studies, we examine the propagation of biases in several widely-used NLP models and its effect on the overall retrieval accuracy. Our work highlights the susceptibility of large, complex models to propagating the biases from human-selected input, which may lead to a deterioration of retrieval accuracy, and the importance of controlling for these biases. Finally, as a way to mitigate the bias, we propose to learn a text representation that is invariant to political ideology while still judging topic relevance.