Designing Fair Ranking Schemes

TL;DR

This paper presents a system that helps users select ranking weights to achieve fairness, using efficient methods to identify and suggest modifications in ranking functions based on fairness criteria, with real-time responsiveness.

Contribution

We introduce a novel approach to identify and modify ranking functions to satisfy fairness criteria efficiently, enabling user-controlled fair ranking adjustments.

Findings

System achieves sub-second response times.

Effectively finds ranking solutions satisfying fairness criteria.

Demonstrates efficiency on real datasets.

Abstract

Items from a database are often ranked based on a combination of multiple criteria. A user may have the flexibility to accept combinations that weigh these criteria differently, within limits. On the other hand, this choice of weights can greatly affect the fairness of the produced ranking. In this paper, we develop a system that helps users choose criterion weights that lead to greater fairness. We consider ranking functions that compute the score of each item as a weighted sum of (numeric) attribute values, and then sort items on their score. Each ranking function can be expressed as a vector of weights, or as a point in a multi-dimensional space. For a broad range of fairness criteria, we show how to efficiently identify regions in this space that satisfy these criteria. Using this identification method, our system is able to tell users whether their proposed ranking function satisfies the desired fairness criteria and, if it does not, to suggest the smallest modification that does. We develop user-controllable approximation that and indexing techniques that are applied during preprocessing, and support sub-second response times during the online phase. Our extensive experiments on real datasets demonstrate that our methods are able to find solutions that satisfy fairness criteria effectively and efficiently.