SoK: What Makes Private Learning Unfair?

TL;DR

This paper surveys how differential privacy in machine learning can unintentionally worsen disparities across groups, analyzing contributing factors and their roles within the training process.

Contribution

It provides the first comprehensive taxonomy of factors influencing fairness disparities in private learning and analyzes their causal impact.

Findings

Dataset and distribution factors are key in disparity occurrence

Differential privacy mechanisms can amplify existing inequalities

Understanding these factors is crucial for mitigation strategies

Abstract

Differential privacy has emerged as the most studied framework for privacy-preserving machine learning. However, recent studies show that enforcing differential privacy guarantees can not only significantly degrade the utility of the model, but also amplify existing disparities in its predictive performance across demographic groups. Although there is extensive research on the identification of factors that contribute to this phenomenon, we still lack a complete understanding of the mechanisms through which differential privacy exacerbates disparities. The literature on this problem is muddled by varying definitions of fairness, differential privacy mechanisms, and inconsistent experimental settings, often leading to seemingly contradictory results. This survey provides the first comprehensive overview of the factors that contribute to the disparate effect of training models with differential privacy guarantees. We discuss their impact and analyze their causal role in such a disparate effect. Our analysis is guided by a taxonomy that categorizes these factors by their position within the machine learning pipeline, allowing us to draw conclusions about their interaction and the feasibility of potential mitigation strategies. We find that factors related to the training dataset and the underlying distribution play a decisive role in the occurrence of disparate impact, highlighting the need for research on these factors to address the issue.