Ex-Ante Assessment of Discrimination in Dataset

TL;DR

This paper introduces FORESEE, a decision tree ensemble method that identifies data subsets where outcomes differ across demographic groups, helping to assess and mitigate bias in machine learning models.

Contribution

The paper presents a novel algorithm, FORESEE, for detecting and characterizing potentially biased samples related to sensitive attributes in datasets.

Findings

FORESEE effectively identifies individuals likely to be misclassified across various classifiers.

The approach helps stakeholders understand and quantify discrimination risks in data.

It can estimate the bias risk of new, unseen samples.

Abstract

Data owners face increasing liability for how the use of their data could harm under-priviliged communities. Stakeholders would like to identify the characteristics of data that lead to algorithms being biased against any particular demographic groups, for example, defined by their race, gender, age, and/or religion. Specifically, we are interested in identifying subsets of the feature space where the ground truth response function from features to observed outcomes differs across demographic groups. To this end, we propose FORESEE, a FORESt of decision trEEs algorithm, which generates a score that captures how likely an individual's response varies with sensitive attributes. Empirically, we find that our approach allows us to identify the individuals who are most likely to be misclassified by several classifiers, including Random Forest, Logistic Regression, Support Vector Machine, and k-Nearest Neighbors. The advantage of our approach is that it allows stakeholders to characterize risky samples that may contribute to discrimination, as well as, use the FORESEE to estimate the risk of upcoming samples.