Projected Boosting with Fairness Constraints: Quantifying the Cost of Fair Training Distributions

TL;DR

This paper introduces FairBoost, a boosting algorithm that incorporates fairness constraints by projecting distributions, and quantifies the tradeoff between fairness and accuracy through theoretical bounds and empirical validation.

Contribution

FairBoost is a novel boosting method that enforces fairness constraints via distribution projection, providing theoretical analysis of the fairness-accuracy tradeoff.

Findings

Theoretical bounds relate fairness constraints to boosting convergence rates.

Experiments show competitive fairness-accuracy tradeoffs.

Stable training curves under fairness constraints.

Abstract

Boosting algorithms enjoy strong theoretical guarantees: when weak learners maintain positive edge, AdaBoost achieves geometric decrease of exponential loss. We study how to incorporate group fairness constraints into boosting while preserving analyzable training dynamics. Our approach, FairBoost, projects the ensemble-induced exponential-weights distribution onto a convex set of distributions satisfying fairness constraints (as a reweighting surrogate), then trains weak learners on this fair distribution. The key theoretical insight is that projecting the training distribution reduces the effective edge of weak learners by a quantity controlled by the KL-divergence of the projection. We prove an exponential-loss bound where the convergence rate depends on weak learner edge minus a "fairness cost" term $\delta_t = \sqrt{\mathrm{KL}(w^t \| q^t)/2}$. This directly quantifies the accuracy-fairness tradeoff in boosting dynamics. Experiments on standard benchmarks validate the theoretical predictions and demonstrate competitive fairness-accuracy tradeoffs with stable training curves.