A fuzzy-rough uncertainty measure to discover bias encoded explicitly or implicitly in features of structured pattern classification datasets

TL;DR

This paper introduces an extended fuzzy-rough uncertainty measure to detect both explicit and implicit bias in structured datasets, aiding fair decision-making without relying on prediction models.

Contribution

It extends previous bias measurement methods by analyzing implicit bias via attribute correlation and provides guidance on fuzzy operators and distance functions.

Findings

The measure can detect bias in protected and non-protected features.

Four bias scenarios are identified for expert evaluation.

Sensitivity analysis identifies optimal fuzzy operators and distance functions.

Abstract

The need to measure bias encoded in tabular data that are used to solve pattern recognition problems is widely recognized by academia, legislators and enterprises alike. In previous work, we proposed a bias quantification measure, called fuzzy-rough uncer-tainty, which relies on the fuzzy-rough set theory. The intuition dictates that protected features should not change the fuzzy-rough boundary regions of a decision class significantly. The extent to which this happens is a proxy for bias expressed as uncertainty in adecision-making context. Our measure's main advantage is that it does not depend on any machine learning prediction model but adistance function. In this paper, we extend our study by exploring the existence of bias encoded implicitly in non-protected featuresas defined by the correlation between protected and unprotected attributes. This analysis leads to four scenarios that domain experts should evaluate before deciding how to tackle bias. In addition, we conduct a sensitivity analysis to determine the fuzzy operatorsand distance function that best capture change in the boundary regions.