You can't see what you can't see: Experimental evidence for how much relevant information may be missed due to Google's Web search personalisation

TL;DR

This study empirically demonstrates that up to 20% of relevant information can be missed due to Google's Web search personalisation, highlighting a significant risk for professional decision-making and the lack of user awareness about this issue.

Contribution

It provides the first empirical evidence quantifying information loss caused by search personalisation and highlights the need for awareness and mitigation strategies among users.

Findings

Up to 20% of relevant information may be missed due to personalisation.

Most users are unaware of the potential biases and risks.

Lack of strategies to mitigate information loss.

Abstract

The influence of Web search personalisation on professional knowledge work is an understudied area. Here we investigate how public sector officials self-assess their dependency on the Google Web search engine, whether they are aware of the potential impact of algorithmic biases on their ability to retrieve all relevant information, and how much relevant information may actually be missed due to Web search personalisation. We find that the majority of participants in our experimental study are neither aware that there is a potential problem nor do they have a strategy to mitigate the risk of missing relevant information when performing online searches. Most significantly, we provide empirical evidence that up to 20% of relevant information may be missed due to Web search personalisation. This work has significant implications for Web research by public sector professionals, who should be provided with training about the potential algorithmic biases that may affect their judgments and decision making, as well as clear guidelines how to minimise the risk of missing relevant information.