Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems

TL;DR

This paper investigates how fairness metrics in multi-component recommender systems can be understood and improved, providing theoretical conditions for fairness composition and practical insights from real-world data.

Contribution

It introduces a theoretical framework for compositional fairness in recommender systems and demonstrates how improving individual components can enhance overall fairness.

Findings

Fairness does not always compose in multi-component systems.

Conditions are identified under which fairness of individual models guarantees system fairness.

Improving fairness in key components significantly impacts overall system fairness.

Abstract

How can we build recommender systems to take into account fairness? Real-world recommender systems are often composed of multiple models, built by multiple teams. However, most research on fairness focuses on improving fairness in a single model. Further, recent research on classification fairness has shown that combining multiple "fair" classifiers can still result in an "unfair" classification system. This presents a significant challenge: how do we understand and improve fairness in recommender systems composed of multiple components? In this paper, we study the compositionality of recommender fairness. We consider two recently proposed fairness ranking metrics: equality of exposure and pairwise ranking accuracy. While we show that fairness in recommendation is not guaranteed to compose, we provide theory for a set of conditions under which fairness of individual models does compose. We then present an analytical framework for both understanding whether a real system's signals can achieve compositional fairness, and improving which component would have the greatest impact on the fairness of the overall system. In addition to the theoretical results, we find on multiple datasets -- including a large-scale real-world recommender system -- that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.