Equity of Attention: Amortizing Individual Fairness in Rankings

TL;DR

This paper introduces measures and mechanisms to ensure individual fairness in rankings by amortizing attention over multiple rankings, addressing inherent position bias and improving fairness without sacrificing ranking quality.

Contribution

It proposes a novel approach to measure and achieve amortized individual fairness in rankings, focusing on individual subjects rather than groups, and formulates it as an online optimization problem.

Findings

Unfair attention distribution in rankings can be significant.

The proposed method improves individual fairness.

High ranking quality can be maintained while enhancing fairness.

Abstract

Rankings of people and items are at the heart of selection-making, match-making, and recommender systems, ranging from employment sites to sharing economy platforms. As ranking positions influence the amount of attention the ranked subjects receive, biases in rankings can lead to unfair distribution of opportunities and resources, such as jobs or income. This paper proposes new measures and mechanisms to quantify and mitigate unfairness from a bias inherent to all rankings, namely, the position bias, which leads to disproportionately less attention being paid to low-ranked subjects. Our approach differs from recent fair ranking approaches in two important ways. First, existing works measure unfairness at the level of subject groups while our measures capture unfairness at the level of individual subjects, and as such subsume group unfairness. Second, as no single ranking can achieve individual attention fairness, we propose a novel mechanism that achieves amortized fairness, where attention accumulated across a series of rankings is proportional to accumulated relevance. We formulate the challenge of achieving amortized individual fairness subject to constraints on ranking quality as an online optimization problem and show that it can be solved as an integer linear program. Our experimental evaluation reveals that unfair attention distribution in rankings can be substantial, and demonstrates that our method can improve individual fairness while retaining high ranking quality.