Certifying Robustness to Programmable Data Bias in Decision Trees

TL;DR

This paper introduces a method to certify that decision trees remain robust against various potential dataset biases, ensuring consistent predictions across many biased datasets, which enhances fairness and reliability.

Contribution

We develop a symbolic technique to certify decision trees' robustness against a wide range of dataset biases, allowing for interpretable and bias-aware model validation.

Findings

Successfully certifies robustness for multiple bias models

Applicable to datasets in fairness literature

Demonstrates viability on real-world bias scenarios

Abstract

Datasets can be biased due to societal inequities, human biases, under-representation of minorities, etc. Our goal is to certify that models produced by a learning algorithm are pointwise-robust to potential dataset biases. This is a challenging problem: it entails learning models for a large, or even infinite, number of datasets, ensuring that they all produce the same prediction. We focus on decision-tree learning due to the interpretable nature of the models. Our approach allows programmatically specifying bias models across a variety of dimensions (e.g., missing data for minorities), composing types of bias, and targeting bias towards a specific group. To certify robustness, we use a novel symbolic technique to evaluate a decision-tree learner on a large, or infinite, number of datasets, certifying that each and every dataset produces the same prediction for a specific test point. We evaluate our approach on datasets that are commonly used in the fairness literature, and demonstrate our approach's viability on a range of bias models.