hyperFA*IR: A hypergeometric approach to fair rankings with finite candidate pool

TL;DR

hyperFA*IR introduces a hypergeometric-based fairness framework for rankings from finite candidate pools, addressing limitations of existing models by accurately capturing sampling without replacement and enabling fair ranking enforcement.

Contribution

It presents a novel hypergeometric approach for fairness in rankings, improving assessment accuracy and operational efficiency over traditional binomial models.

Findings

More accurate fairness assessment in small or marginalised groups.

Efficient Monte Carlo algorithm for unfair ranking detection.

Effective adaptation for affirmative action policies.

Abstract

Ranking algorithms play a pivotal role in decision-making processes across diverse domains, from search engines to job applications. When rankings directly impact individuals, ensuring fairness becomes essential, particularly for groups that are marginalised or misrepresented in the data. Most of the existing group fairness frameworks often rely on ensuring proportional representation of protected groups. However, these approaches face limitations in accounting for the stochastic nature of ranking processes or the finite size of candidate pools. To this end, we present hyperFA*IR, a framework for assessing and enforcing fairness in rankings drawn from a finite set of candidates. It relies on a generative process based on the hypergeometric distribution, which models real-world scenarios by sampling without replacement from fixed group sizes. This approach improves fairness assessment when top-$k$ selections are large relative to the pool or when protected groups are small. We compare our approach to the widely used binomial model, which treats each draw as independent with fixed probability, and demonstrate$-$both analytically and empirically$-$that our method more accurately reproduces the statistical properties of sampling from a finite population. To operationalise this framework, we propose a Monte Carlo-based algorithm that efficiently detects unfair rankings by avoiding computationally expensive parameter tuning. Finally, we adapt our generative approach to define affirmative action policies by introducing weights into the sampling process.