Fairness of Exposure in Light of Incomplete Exposure Estimation

TL;DR

This paper addresses fairness of exposure in ranking systems when exposure estimates are incomplete, proposing FELIX to improve fairness without sacrificing utility, especially in top-k rankings.

Contribution

Introduces FELIX, a method to avoid unknown exposure rankings without utility or fairness loss, and extends fairness analysis to top-k ranking scenarios.

Findings

FELIX reduces unknown exposure rankings significantly.

FELIX maintains user utility comparable to existing methods.

FELIX improves fairness in top-k ranking settings.

Abstract

Fairness of exposure is a commonly used notion of fairness for ranking systems. It is based on the idea that all items or item groups should get exposure proportional to the merit of the item or the collective merit of the items in the group. Often, stochastic ranking policies are used to ensure fairness of exposure. Previous work unrealistically assumes that we can reliably estimate the expected exposure for all items in each ranking produced by the stochastic policy. In this work, we discuss how to approach fairness of exposure in cases where the policy contains rankings of which, due to inter-item dependencies, we cannot reliably estimate the exposure distribution. In such cases, we cannot determine whether the policy can be considered fair. Our contributions in this paper are twofold. First, we define a method called FELIX for finding stochastic policies that avoid showing rankings with unknown exposure distribution to the user without having to compromise user utility or item fairness. Second, we extend the study of fairness of exposure to the top-k setting and also assess FELIX in this setting. We find that FELIX can significantly reduce the number of rankings with unknown exposure distribution without a drop in user utility or fairness compared to existing fair ranking methods, both for full-length and top-k rankings. This is an important first step in developing fair ranking methods for cases where we have incomplete knowledge about the user's behaviour.