Quantitative Verification of Fairness in Tree Ensembles

TL;DR

This paper introduces an efficient method for quantitatively verifying fairness in tree ensembles, providing bounds on counterexamples and outperforming existing fairness testing techniques.

Contribution

It extends quantitative verification to tree ensembles, offering an efficient, model-agnostic technique that delivers both upper and lower bounds on counterexamples.

Findings

Effective bounds on counterexamples in tree ensembles

Outperforms state-of-the-art fairness testing methods

Demonstrates efficiency on multiple datasets

Abstract

This work focuses on quantitative verification of fairness in tree ensembles. Unlike traditional verification approaches that merely return a single counterexample when the fairness is violated, quantitative verification estimates the ratio of all counterexamples and characterizes the regions where they occur, which is important information for diagnosing and mitigating bias. To date, quantitative verification has been explored almost exclusively for deep neural networks (DNNs). Representative methods, such as DeepGemini and FairQuant, all build on the core idea of Counterexample-Guided Abstraction Refinement, a generic framework that could be adapted to other model classes. We extended the framework into a model-agnostic form, but discovered two limitations: (i) it can provide only lower bounds, and (ii) its performance scales poorly. Exploiting the discrete structure of tree ensembles, our work proposes an efficient quantification technique that delivers any-time upper and lower bounds. Experiments on five widely used datasets demonstrate its effectiveness and efficiency. When applied to fairness testing, our quantification method significantly outperforms state-of-the-art testing techniques.