Newsalyze: Effective Communication of Person-Targeting Biases in News Articles

TL;DR

This paper introduces a system for identifying and visualizing biases in news articles, and demonstrates through a large-scale user study that bias-sensitive visualizations significantly increase bias-awareness among consumers.

Contribution

It presents a novel bias detection system, bias-sensitive visualizations, and a comprehensive user study measuring their impact on bias-awareness in news consumption.

Findings

Bias-sensitive overviews increase bias-awareness significantly.

Content-driven bias detection groups similar biased articles.

Visualizations effectively communicate bias to non-experts.

Abstract

Media bias and its extreme form, fake news, can decisively affect public opinion. Especially when reporting on policy issues, slanted news coverage may strongly influence societal decisions, e.g., in democratic elections. Our paper makes three contributions to address this issue. First, we present a system for bias identification, which combines state-of-the-art methods from natural language understanding. Second, we devise bias-sensitive visualizations to communicate bias in news articles to non-expert news consumers. Third, our main contribution is a large-scale user study that measures bias-awareness in a setting that approximates daily news consumption, e.g., we present respondents with a news overview and individual articles. We not only measure the visualizations' effect on respondents' bias-awareness, but we can also pinpoint the effects on individual components of the visualizations by employing a conjoint design. Our bias-sensitive overviews strongly and significantly increase bias-awareness in respondents. Our study further suggests that our content-driven identification method detects groups of similarly slanted news articles due to substantial biases present in individual news articles. In contrast, the reviewed prior work rather only facilitates the visibility of biases, e.g., by distinguishing left- and right-wing outlets.