Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media

TL;DR

This paper presents a framework to quantify and distinguish between data-driven and algorithmic biases in political search results on social media, revealing both sources significantly influence bias in Twitter search outcomes.

Contribution

It introduces a novel framework for separating and quantifying data and algorithm biases in social media search results, specifically applied to political queries on Twitter.

Findings

Both input data and ranking algorithms significantly contribute to search bias.

Bias varies across different political topics and perspectives.

Signaling bias could improve transparency in social media search results.

Abstract

Search systems in online social media sites are frequently used to find information about ongoing events and people. For topics with multiple competing perspectives, such as political events or political candidates, bias in the top ranked results significantly shapes public opinion. However, bias does not emerge from an algorithm alone. It is important to distinguish between the bias that arises from the data that serves as the input to the ranking system and the bias that arises from the ranking system itself. In this paper, we propose a framework to quantify these distinct biases and apply this framework to politics-related queries on Twitter. We found that both the input data and the ranking system contribute significantly to produce varying amounts of bias in the search results and in different ways. We discuss the consequences of these biases and possible mechanisms to signal this bias in social media search systems' interfaces.