Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training

TL;DR

This paper provides a theoretical analysis of how bias evolves during stochastic gradient descent training in machine learning models, revealing the influence of data sub-populations and feature properties on bias dynamics.

Contribution

It offers an exact high-dimensional analytical description of bias dynamics in a teacher-student model, bridging the gap in understanding transient bias formation during training.

Findings

Bias shifts during training influenced by sub-population properties

Heterogeneous data can generate and amplify bias over time

Empirical validation on complex datasets supports theoretical insights

Abstract

Machine learning systems often acquire biases by leveraging undesired features in the data, impacting accuracy variably across different sub-populations. Current understanding of bias formation mostly focuses on the initial and final stages of learning, leaving a gap in knowledge regarding the transient dynamics. To address this gap, this paper explores the evolution of bias in a teacher-student setup modeling different data sub-populations with a Gaussian-mixture model. We provide an analytical description of the stochastic gradient descent dynamics of a linear classifier in this setting, which we prove to be exact in high dimension. Notably, our analysis reveals how different properties of sub-populations influence bias at different timescales, showing a shifting preference of the classifier during training. Applying our findings to fairness and robustness, we delineate how and when heterogeneous data and spurious features can generate and amplify bias. We empirically validate our results in more complex scenarios by training deeper networks on synthetic and real datasets, including CIFAR10, MNIST, and CelebA.