Fairness in Ranking: A Survey

TL;DR

This survey provides a comprehensive overview of fairness in ranking algorithms, connecting formalizations, techniques, and evaluation methods across multiple subfields to guide future research and application.

Contribution

It introduces a unified narrative on fairness frameworks, classification of intervention methods, and evaluation strategies in ranking systems, synthesizing recent advances.

Findings

Unified fairness frameworks across subfields

Classification of fairness interventions

Evaluation datasets and metrics for fair ranking

Abstract

In the past few years, there has been much work on incorporating fairness requirements into algorithmic rankers, with contributions coming from the data management, algorithms, information retrieval, and recommender systems communities. In this survey we give a systematic overview of this work, offering a broad perspective that connects formalizations and algorithmic approaches across subfields. An important contribution of our work is in developing a common narrative around the value frameworks that motivate specific fairness-enhancing interventions in ranking. This allows us to unify the presentation of mitigation objectives and of algorithmic techniques to help meet those objectives or identify trade-offs. In this survey, we describe four classification frameworks for fairness-enhancing interventions, along which we relate the technical methods surveyed in this paper, discuss evaluation datasets, and present technical work on fairness in score-based ranking. Then, we present methods that incorporate fairness in supervised learning, and also give representative examples of recent work on fairness in recommendation and matchmaking systems. We also discuss evaluation frameworks for fair score-based ranking and fair learning-to-rank, and draw a set of recommendations for the evaluation of fair ranking methods.