Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics

TL;DR

This paper evaluates how ranking fairness metrics can be used to measure viewpoint diversity in search results, providing insights into their interpretation and suitability for different scenarios.

Contribution

It introduces a framework for assessing viewpoint diversity using ranking fairness metrics and compares their effectiveness through a controlled simulation study.

Findings

Ranking fairness metrics can effectively measure viewpoint diversity.

Different metrics are suitable depending on the specific context.

The study guides future research in evaluating diversity in real search rankings.

Abstract

The way pages are ranked in search results influences whether the users of search engines are exposed to more homogeneous, or rather to more diverse viewpoints. However, this viewpoint diversity is not trivial to assess. In this paper we use existing and novel ranking fairness metrics to evaluate viewpoint diversity in search result rankings. We conduct a controlled simulation study that shows how ranking fairness metrics can be used for viewpoint diversity, how their outcome should be interpreted, and which metric is most suitable depending on the situation. This paper lays out important ground work for future research to measure and assess viewpoint diversity in real search result rankings.