Does Machine Bring in Extra Bias in Learning? Approximating Fairness in Models Promptly

TL;DR

This paper introduces a novel manifold-based fairness measure for ML models, along with an efficient approximation algorithm, addressing the challenge of evaluating discrimination without high computational costs.

Contribution

It proposes a new harmonic fairness measure based on manifolds and an approximation algorithm for practical discrimination assessment in ML models.

Findings

HFM is a valid fairness measure for classifiers.

ApproxDist effectively estimates distances between sets.

The methods are efficient and applicable in real-world scenarios.

Abstract

Providing various machine learning (ML) applications in the real world, concerns about discrimination hidden in ML models are growing, particularly in high-stakes domains. Existing techniques for assessing the discrimination level of ML models include commonly used group and individual fairness measures. However, these two types of fairness measures are usually hard to be compatible with each other, and even two different group fairness measures might be incompatible as well. To address this issue, we investigate to evaluate the discrimination level of classifiers from a manifold perspective and propose a "harmonic fairness measure via manifolds (HFM)" based on distances between sets. Yet the direct calculation of distances might be too expensive to afford, reducing its practical applicability. Therefore, we devise an approximation algorithm named "Approximation of distance between sets (ApproxDist)" to facilitate accurate estimation of distances, and we further demonstrate its algorithmic effectiveness under certain reasonable assumptions. Empirical results indicate that the proposed fairness measure HFM is valid and that the proposed ApproxDist is effective and efficient.