# Fairness in Recommendation Ranking through Pairwise Comparisons

## TL;DR

This paper introduces novel pairwise comparison metrics and a regularizer to improve fairness in recommendation rankings, demonstrating significant fairness improvements in a large-scale production system.

## Contribution

It proposes new metrics and a regularization method based on pairwise comparisons to enhance fairness in recommender system rankings.

## Key findings

- Improved pairwise fairness in a large-scale recommender system
- New metrics for evaluating fairness based on pairwise comparisons
- Effective regularizer for training fairer ranking models

## Abstract

Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information. As such it is important to ask: what are the possible fairness risks, how can we quantify them, and how should we address them? In this paper we offer a set of novel metrics for evaluating algorithmic fairness concerns in recommender systems. In particular we show how measuring fairness based on pairwise comparisons from randomized experiments provides a tractable means to reason about fairness in rankings from recommender systems. Building on this metric, we offer a new regularizer to encourage improving this metric during model training and thus improve fairness in the resulting rankings. We apply this pairwise regularization to a large-scale, production recommender system and show that we are able to significantly improve the system's pairwise fairness.