Auditing the Biases Enacted by YouTube for Political Topics in Germany

TL;DR

This paper investigates biases in YouTube's recommendation system regarding political topics in Germany, revealing popularity bias and emotional shifts, and emphasizes audits as tools for accountability.

Contribution

It introduces audit methodologies to evaluate biases in YouTube's political content recommendations and analyzes emotional and popularity biases in the system.

Findings

YouTube recommends increasingly popular but unrelated videos.

Emotional tone shifts towards more happiness and less sadness.

Popularity bias correlates with emotional content in recommendations.

Abstract

With YouTube's growing importance as a news platform, its recommendation system came under increased scrutiny. Recognizing YouTube's recommendation system as a broadcaster of media, we explore the applicability of laws that require broadcasters to give important political, ideological, and social groups adequate opportunity to express themselves in the broadcasted program of the service. We present audits as an important tool to enforce such laws and to ensure that a system operates in the public's interest. To examine whether YouTube is enacting certain biases, we collected video recommendations about political topics by following chains of ten recommendations per video. Our findings suggest that YouTube's recommendation system is enacting important biases. We find that YouTube is recommending increasingly popular but topically unrelated videos. The sadness evoked by the recommended videos decreases while the happiness increases. We discuss the strong popularity bias we identified and analyze the link between the popularity of content and emotions. We also discuss how audits empower researchers and civic hackers to monitor complex machine learning (ML)-based systems like YouTube's recommendation system.