Perception-Aware Bias Detection for Query Suggestions

TL;DR

This paper introduces perception-aware metrics to improve bias detection in web search query suggestions, addressing challenges like sparseness and subliminal perception, and demonstrates enhanced bias detection capabilities.

Contribution

It extends existing bias detection pipelines with perception-aware metrics tailored for query suggestions, enabling more accurate identification of topical bias.

Findings

Enhanced bias detection pipeline detects more systematic topical bias.

Perception-aware metrics reflect biases perceptible to users.

Analysis confirms improved bias detection effectiveness.

Abstract

Bias in web search has been in the spotlight of bias detection research for quite a while. At the same time, little attention has been paid to query suggestions in this regard. Awareness of the problem of biased query suggestions has been raised. Likewise, there is a rising need for automatic bias detection approaches. This paper adds on the bias detection pipeline for bias detection in query suggestions of person-related search developed by Bonart et al. \cite{Bonart_2019a}. The sparseness and lack of contextual metadata of query suggestions make them a difficult subject for bias detection. Furthermore, query suggestions are perceived very briefly and subliminally. To overcome these issues, perception-aware metrics are introduced. Consequently, the enhanced pipeline is able to better detect systematic topical bias in search engine query suggestions for person-related searches. The results of an analysis performed with the developed pipeline confirm this assumption. Due to the perception-aware bias detection metrics, findings produced by the pipeline can be assumed to reflect bias that users would discern.