Towards Analyzing the Bias of News Recommender Systems Using Sentiment and Stance Detection

TL;DR

This paper investigates biases in news recommender systems by analyzing sentiment and stance, revealing that most models tend to favor negative and anti-migration content, thus potentially reinforcing user biases and reducing diversity.

Contribution

It introduces a method to quantify bias in news recommenders using stance and sentiment analysis, and compares four models on a German migration news corpus.

Findings

All models show a bias towards negative sentiments and anti-migration stances.

Recommender systems tend to amplify preexisting user biases.

Knowledge-aware models are less biased but less accurate.

Abstract

News recommender systems are used by online news providers to alleviate information overload and to provide personalized content to users. However, algorithmic news curation has been hypothesized to create filter bubbles and to intensify users' selective exposure, potentially increasing their vulnerability to polarized opinions and fake news. In this paper, we show how information on news items' stance and sentiment can be utilized to analyze and quantify the extent to which recommender systems suffer from biases. To that end, we have annotated a German news corpus on the topic of migration using stance detection and sentiment analysis. In an experimental evaluation with four different recommender systems, our results show a slight tendency of all four models for recommending articles with negative sentiments and stances against the topic of refugees and migration. Moreover, we observed a positive correlation between the sentiment and stance bias of the text-based recommenders and the preexisting user bias, which indicates that these systems amplify users' opinions and decrease the diversity of recommended news. The knowledge-aware model appears to be the least prone to such biases, at the cost of predictive accuracy.