Differentiable Optimization Layers for Guaranteed Fairness in Deep Learning

TL;DR

This paper introduces a differentiable fairness layer for neural networks that guarantees output parity and proposes an online primal-dual algorithm for streaming data, with theoretical and empirical validation.

Contribution

It presents a novel fairness layer integrated into neural networks and an online algorithm that ensures fairness guarantees in streaming predictions.

Findings

The fairness layer guarantees output parity in neural networks.

The online primal-dual algorithm provides provable fairness in streaming settings.

Numerical experiments validate the effectiveness of the proposed methods.

Abstract

Differentiable optimization layers are traditionally integrated in predict-then-optimize frameworks where a neural model estimates parameters that subsequently serve as fixed inputs to downstream decision-making optimization problems. In this work, we introduce the concept of a "fairness layer": a differentiable optimization layer appended to a model's output layer that guarantees a chosen notion of output parity is satisfied when integrated into a neural network. Additionally, we introduce an online primal-dual inference algorithm that provides provable aggregate fairness guarantees for streaming predictions with arbitrarily small batch sizes, where traditional per-batch constraints become overly restrictive. Numerical experiments demonstrate the effectiveness of the fairness layer and associated algorithm, and theoretical analysis characterizes the layer's differentiability and stability properties during model training and backpropagation. Our code for these experiments is publicly available on GitHub (https://github.com/dtroxell19/FairDL-ICML-2026.git) and our public Python package documentation can be found online: https://dtroxell19.github.io/fairness_training/.