Optimizing Group-Fair Plackett-Luce Ranking Models for Relevance and Ex-Post Fairness

TL;DR

This paper introduces a novel group-fair Plackett-Luce ranking model that optimizes relevance while ensuring ex-post fairness, outperforming existing methods in relevance and fairness on real datasets.

Contribution

It proposes a new objective and a group-fair Plackett-Luce model that directly incorporate ex-post fairness constraints into the ranking optimization process.

Findings

Guarantees fairness alongside improved relevance compared to baselines.

Outperforms post-processing methods in relevance while maintaining fairness.

Handles implicit bias in training data effectively.

Abstract

In learning-to-rank (LTR), optimizing only the relevance (or the expected ranking utility) can cause representational harm to certain categories of items. Moreover, if there is implicit bias in the relevance scores, LTR models may fail to optimize for true relevance. Previous works have proposed efficient algorithms to train stochastic ranking models that achieve fairness of exposure to the groups ex-ante (or, in expectation), which may not guarantee representation fairness to the groups ex-post, that is, after realizing a ranking from the stochastic ranking model. Typically, ex-post fairness is achieved by post-processing, but previous work does not train stochastic ranking models that are aware of this post-processing. In this paper, we propose a novel objective that maximizes expected relevance only over those rankings that satisfy given representation constraints to ensure ex-post fairness. Building upon recent work on an efficient sampler for ex-post group-fair rankings, we propose a group-fair Plackett-Luce model and show that it can be efficiently optimized for our objective in the LTR framework. Experiments on three real-world datasets show that our group-fair algorithm guarantees fairness alongside usually having better relevance compared to the LTR baselines. In addition, our algorithm also achieves better relevance than post-processing baselines, which also ensures ex-post fairness. Further, when implicit bias is injected into the training data, our algorithm typically outperforms existing LTR baselines in relevance.