An Effective Theory of Bias Amplification

TL;DR

This paper develops a rigorous theoretical framework for understanding bias amplification in machine learning, specifically in ridge regression models, revealing how model choices and data properties influence bias and test performance across groups.

Contribution

It provides the first precise analytical theory explaining bias amplification phenomena in ridge regression models, connecting model design, data distribution, and bias outcomes.

Findings

Optimal regularization can prevent bias amplification

Test error disparities between groups can persist despite increased parameters

Theory aligns with empirical observations on synthetic datasets

Abstract

Machine learning models can capture and amplify biases present in data, leading to disparate test performance across social groups. To better understand, evaluate, and mitigate these biases, a deeper theoretical understanding of how model design choices and data distribution properties contribute to bias is needed. In this work, we contribute a precise analytical theory in the context of ridge regression, both with and without random projections, where the former models feedforward neural networks in a simplified regime. Our theory offers a unified and rigorous explanation of machine learning bias, providing insights into phenomena such as bias amplification and minority-group bias in various feature and parameter regimes. For example, we observe that there may be an optimal regularization penalty or training time to avoid bias amplification, and there can be differences in test error between groups that are not alleviated with increased parameterization. Importantly, our theoretical predictions align with empirical observations reported in the literature on machine learning bias. We extensively empirically validate our theory on synthetic and semi-synthetic datasets.