Reducing Disparate Exposure in Ranking: A Learning To Rank Approach

TL;DR

This paper introduces DELTR, an in-processing learning-to-rank framework designed to reduce exposure disparities and discrimination in search rankings by optimizing relevance and fairness during training.

Contribution

The paper proposes DELTR, a novel in-processing method for ranking that directly addresses bias and inequality at training time, outperforming existing pre- and post-processing approaches.

Findings

DELTR effectively reduces exposure disparities in rankings.

Being 'colorblind' can either improve or worsen relevance and fairness depending on bias.

DELTR outperforms existing methods across various scenarios.

Abstract

Ranked search results have become the main mechanism by which we find content, products, places, and people online. Thus their ordering contributes not only to the satisfaction of the searcher, but also to career and business opportunities, educational placement, and even social success of those being ranked. Researchers have become increasingly concerned with systematic biases in data-driven ranking models, and various post-processing methods have been proposed to mitigate discrimination and inequality of opportunity. This approach, however, has the disadvantage that it still allows an unfair ranking model to be trained. In this paper we explore a new in-processing approach: DELTR, a learning-to-rank framework that addresses potential issues of discrimination and unequal opportunity in rankings at training time. We measure these problems in terms of discrepancies in the average group exposure and design a ranker that optimizes search results in terms of relevance and in terms of reducing such discrepancies. We perform an extensive experimental study showing that being "colorblind" can be among the best or the worst choices from the perspective of relevance and exposure, depending on how much and which kind of bias is present in the training set. We show that our in-processing method performs better in terms of relevance and exposure than a pre-processing and a post-processing method across all tested scenarios.