Finite-Sample and Distribution-Free Fair Classification: Optimal Trade-off Between Excess Risk and Fairness, and the Cost of Group-Blindness

TL;DR

This paper introduces a distribution-free, finite-sample fair classification framework that balances excess risk and fairness, applicable to both group-aware and group-blind settings, with theoretical guarantees and empirical validation.

Contribution

It proposes a unified, post-processing fair classification method with optimal theoretical guarantees, applicable without distributional assumptions, and demonstrates its effectiveness empirically.

Findings

The framework provides distribution-free fairness guarantees.

The proposed algorithm is minimax rate-optimal up to logarithmic factors.

Empirical results show competitive or superior performance compared to existing methods.

Abstract

Algorithmic fairness has become a central concern in modern machine learning and AI applications. However, two pressing challenges remain: (1) The fairness guarantees of existing methods often rely on specific data distributional assumptions and large sample sizes, which can lead to fairness violations in practice. (2) Due to legal and societal considerations, using sensitive group attributes during decision-making (referred to as the group-blind setting) may not always be feasible. In this work, we quantify the impact of enforcing algorithmic fairness and group-blindness/awareness in binary classification under group fairness constraints. Specifically, we propose a unified framework for fair classification that provides distribution-free and finite-sample fairness guarantees with controlled excess risk. This framework is applicable to various group fairness notions in both group-aware and group-blind scenarios. Our approach is based on a post-processing procedure that can be applied to arbitrary black-box models, making it directly compatible with modern machine learning pipelines. Furthermore, we establish a minimax lower bound showing the minimax rate-optimality of our proposed algorithm up to logarithmic factors. Through extensive synthetic and real data studies, we further demonstrate the competitive or superior performance of our algorithm compared to existing methods, and provide empirical support for our theoretical findings.