Fairness Dynamics During Training

TL;DR

This paper studies how biases in large language models evolve during training, introduces new metrics for fairness, and demonstrates how early stopping can improve fairness without significantly harming performance.

Contribution

It introduces two novel metrics to monitor fairness dynamics during training and shows how early stopping can mitigate biases in large language models.

Findings

Biases can emerge suddenly during training.

Early stopping can significantly improve fairness.

Larger models tend to exhibit more biases.

Abstract

We investigate fairness dynamics during Large Language Model (LLM) training to enable the diagnoses of biases and mitigations through training interventions like early stopping; we find that biases can emerge suddenly and do not always follow common performance metrics. We introduce two new metrics to evaluate fairness dynamics holistically during model pre-training: Average Rank and Jensen-Shannon Divergence by Parts. These metrics provide insights into the Pythia models' progression of biases in gender prediction of occupations on the WinoBias dataset. By monitoring these dynamics, we find that (1) Pythia-6.9b is biased towards men; it becomes more performant and confident predicting "male" than "female" during training, (2) via early-stopping, Pythia-6.9b can exchange 1.7% accuracy on LAMBADA for a 92.5% increase in fairness, and (3) larger models can exhibit more bias; Pythia-6.9b makes more assumptions about gender than Pythia-160m, even when a subject's gender is not specified.