The Matter of Chance: Auditing Web Search Results Related to the 2020 U.S. Presidential Primary Elections Across Six Search Engines

TL;DR

This study audits six search engines during the 2020 U.S. presidential primaries to reveal significant differences and randomness in search results, which can influence public opinion and create information inequalities.

Contribution

It introduces an algorithmic auditing methodology using virtual agents to systematically analyze and compare search result biases across multiple engines in a controlled setting.

Findings

Search results vary significantly between search engines.

Results for the same query can differ due to inherent randomness.

Search engines prioritize different information sources for candidates.

Abstract

We examine how six search engines filter and rank information in relation to the queries on the U.S. 2020 presidential primary elections under the default - that is nonpersonalized - conditions. For that, we utilize an algorithmic auditing methodology that uses virtual agents to conduct large-scale analysis of algorithmic information curation in a controlled environment. Specifically, we look at the text search results for "us elections", "donald trump", "joe biden" and "bernie sanders" queries on Google, Baidu, Bing, DuckDuckGo, Yahoo, and Yandex, during the 2020 primaries. Our findings indicate substantial differences in the search results between search engines and multiple discrepancies within the results generated for different agents using the same search engine. It highlights that whether users see certain information is decided by chance due to the inherent randomization of search results. We also find that some search engines prioritize different categories of information sources with respect to specific candidates. These observations demonstrate that algorithmic curation of political information can create information inequalities between the search engine users even under nonpersonalized conditions. Such inequalities are particularly troubling considering that search results are highly trusted by the public and can shift the opinions of undecided voters as demonstrated by previous research.