FairGBM: Gradient Boosting with Fairness Constraints

TL;DR

FairGBM introduces a gradient boosting framework that enforces fairness constraints with minimal performance loss and significantly faster training, making fair machine learning more practical for real-world tabular data applications.

Contribution

The paper proposes a novel dual ascent learning framework for GBDT that incorporates fairness constraints efficiently using smooth proxies and a proxy-Lagrangian approach.

Findings

Achieves minimal performance impact compared to unconstrained GBDT.

Provides an order of magnitude speedup in training time.

Enables practical fairness enforcement in high-stakes tabular data applications.

Abstract

Tabular data is prevalent in many high-stakes domains, such as financial services or public policy. Gradient Boosted Decision Trees (GBDT) are popular in these settings due to their scalability, performance, and low training cost. While fairness in these domains is a foremost concern, existing in-processing Fair ML methods are either incompatible with GBDT, or incur in significant performance losses while taking considerably longer to train. We present FairGBM, a dual ascent learning framework for training GBDT under fairness constraints, with little to no impact on predictive performance when compared to unconstrained GBDT. Since observational fairness metrics are non-differentiable, we propose smooth convex error rate proxies for common fairness criteria, enabling gradient-based optimization using a ``proxy-Lagrangian'' formulation. Our implementation shows an order of magnitude speedup in training time relative to related work, a pivotal aspect to foster the widespread adoption of FairGBM by real-world practitioners.