Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems

TL;DR

This study explores how large tech companies' recommender system teams understand and implement fairness, highlighting organizational and technical challenges and providing practical recommendations for integrating fairness into workflows.

Contribution

It provides an empirical analysis of industry practices and challenges in operationalizing fairness in recommender systems through interviews with practitioners.

Findings

Practitioners face challenges defining fairness in RS contexts.

Balancing stakeholder interests is complex in fairness efforts.

Organizational barriers hinder fairness integration, such as time constraints and communication gaps.

Abstract

Recommender systems (RS), which are widely deployed across high-stakes domains, are susceptible to biases that can cause large-scale societal impacts. Researchers have proposed methods to measure and mitigate such biases - but translating academic theory into practice is inherently challenging. Through a semi-structured interview study (N=11), we map the RS practitioner workflow within large technology companies, focusing on how technical teams consider fairness internally and in collaboration with legal, data, and fairness teams. We identify key challenges to incorporating fairness into existing RS workflows: defining fairness in RS contexts, balancing multi-stakeholder interests, and navigating dynamic environments. We also identify key organization-wide challenges: making time for fairness work and facilitating cross-team communication. Finally, we offer actionable recommendations for the RS community, including practitioners and HCI researchers.