Trustworthiness Evaluations of Search Results: The Impact of Rank and Misinformation

TL;DR

This study investigates how rank, misinformation, and warning banners influence users' trust in search results, revealing that higher rank increases clicks but not trust, and warnings can backfire.

Contribution

It provides empirical evidence on the effects of rank, misinformation, and warnings on trust in search results using Covid-19 queries and experiments.

Findings

Higher-ranked results are clicked more but not more trusted.

Misinformation does not reduce trust in accurate results below it.

Warning banners can decrease trust in accurate information.

Abstract

Users rely on search engines for information in critical contexts, such as public health emergencies. Understanding how users evaluate the trustworthiness of search results is therefore essential. Research has identified rank and the presence of misinformation as factors impacting perceptions and click behavior in search. Here, we elaborate on these findings by measuring the effects of rank and misinformation, as well as warning banners, on the perceived trustworthiness of individual results in search. We conducted three online experiments (N=3196) using Covid-19-related queries to address this question. We show that although higher-ranked results are clicked more often, they are not more trusted. We also show that misinformation did not change trust in accurate results below it. However, a warning about unreliable sources backfired, decreasing trust in accurate information but not misinformation. This work addresses concerns about how people evaluate information in search, and illustrates the dangers of generic prevention approaches.