OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning

TL;DR

OmniFair is a versatile, model-agnostic declarative system that enables specifying and enforcing multiple group fairness constraints in machine learning, improving fairness without significant accuracy loss or computational overhead.

Contribution

It introduces a declarative interface supporting all common group fairness notions and multiple constraints simultaneously, with algorithms optimized for accuracy-fairness trade-offs.

Findings

Reduces accuracy loss by up to 94.8% compared to other methods.

Supports all major fairness notions including statistical parity, equalized odds, and predictive parity.

Achieves up to 270 times faster performance than in-processing approaches.

Abstract

Machine learning (ML) is increasingly being used to make decisions in our society. ML models, however, can be unfair to certain demographic groups (e.g., African Americans or females) according to various fairness metrics. Existing techniques for producing fair ML models either are limited to the type of fairness constraints they can handle (e.g., preprocessing) or require nontrivial modifications to downstream ML training algorithms (e.g., in-processing). We propose a declarative system OmniFair for supporting group fairness in ML. OmniFair features a declarative interface for users to specify desired group fairness constraints and supports all commonly used group fairness notions, including statistical parity, equalized odds, and predictive parity. OmniFair is also model-agnostic in the sense that it does not require modifications to a chosen ML algorithm. OmniFair also supports enforcing multiple user declared fairness constraints simultaneously while most previous techniques cannot. The algorithms in OmniFair maximize model accuracy while meeting the specified fairness constraints, and their efficiency is optimized based on the theoretically provable monotonicity property regarding the trade-off between accuracy and fairness that is unique to our system. We conduct experiments on commonly used datasets that exhibit bias against minority groups in the fairness literature. We show that OmniFair is more versatile than existing algorithmic fairness approaches in terms of both supported fairness constraints and downstream ML models. OmniFair reduces the accuracy loss by up to $94.8\%$ compared with the second best method. OmniFair also achieves similar running time to preprocessing methods, and is up to $270\times$ faster than in-processing methods.