Users' Perception of Search Engine Biases and Satisfaction

TL;DR

This study investigates end-users' perceptions of search engine biases and how diversity in search results affects their satisfaction, revealing a preference for relevance and consistency over diversity.

Contribution

It provides insights into user preferences regarding search engine biases and demonstrates the impact of result diversity on user satisfaction from an end-user perspective.

Findings

Participants prefer original Bing pages over diversified results.

Diversity in results can reduce perceived relevance and satisfaction.

Users focus more on top search results than lower ones.

Abstract

Search engines could consistently favor certain values over the others, which is considered as biased due to the built-in infrastructures. Many studies have been dedicated to detect, control, and mitigate the impacts of the biases from the perspectives of search engines themselves. In our study, we take the perspective from end-users to analyze their perceptions of search engine biases and their satisfaction when the biases are regulated. In the study, we paired a real search page from search engine Bing with a synthesized page that has more diversities in the results (i.e. less biased). Both pages show the top-10 search items given search queries and we asked participants which one do they prefer and why do they prefer the one selected. Statistical analyses revealed that overall, participants prefer the original Bing pages and the locations where the diversities are introduced are also associated with users' preferences. We found out that users prefer results that are more consistent and relevant to the search queries. Introducing diversities undermines the relevance of the search results and impairs users' satisfaction to some degree. Additionally, we confirmed that users tend to pay more attention to the top portion of the results than the bottom ones.