From Models to Systems: A Comprehensive Fairness Framework for Compositional Recommender Systems

TL;DR

This paper introduces a comprehensive system-level fairness framework for compositional recommender systems, emphasizing end-user utility and component interactions, moving beyond traditional model-level fairness assessments.

Contribution

It proposes a holistic fairness modeling approach for recommendation systems, incorporating system interactions and optimizing utility and equity jointly.

Findings

System-level fairness modeling improves equity in recommendations.

Joint optimization enhances end-user utility across diverse groups.

Empirical results validate the framework on synthetic and real datasets.

Abstract

Fairness research in machine learning often centers on ensuring equitable performance of individual models. However, real-world recommendation systems are built on multiple models and even multiple stages, from candidate retrieval to scoring and serving, which raises challenges for responsible development and deployment. This system-level view, as highlighted by regulations like the EU AI Act, necessitates moving beyond auditing individual models as independent entities. We propose a holistic framework for modeling system-level fairness, focusing on the end-utility delivered to diverse user groups, and consider interactions between components such as retrieval and scoring models. We provide formal insights on the limitations of focusing solely on model-level fairness and highlight the need for alternative tools that account for heterogeneity in user preferences. To mitigate system-level disparities, we adapt closed-box optimization tools (e.g., BayesOpt) to jointly optimize utility and equity. We empirically demonstrate the effectiveness of our proposed framework on synthetic and real datasets, underscoring the need for a system-level framework.