Beyond Incompatibility: Trade-offs between Mutually Exclusive Fairness Criteria in Machine Learning and Law

TL;DR

This paper introduces FAIM, a novel algorithm that interpolates between conflicting fairness criteria in machine learning, enabling more flexible fairness adjustments while addressing legal and ethical considerations.

Contribution

The paper presents FAIM, a new method for continuously balancing multiple fairness criteria, addressing the incompatibility issue in algorithmic fairness.

Findings

FAIM effectively interpolates fairness measures on synthetic and real-world data.

It demonstrates potential for legal compliance in various domains.

The approach offers a practical tool for balancing fairness trade-offs.

Abstract

Fair and trustworthy AI is becoming ever more important in both machine learning and legal domains. One important consequence is that decision makers must seek to guarantee a 'fair', i.e., non-discriminatory, algorithmic decision procedure. However, there are several competing notions of algorithmic fairness that have been shown to be mutually incompatible under realistic factual assumptions. This concerns, for example, the widely used fairness measures of 'calibration within groups' and 'balance for the positive/negative class'. In this paper, we present a novel algorithm (FAir Interpolation Method: FAIM) for continuously interpolating between these three fairness criteria. Thus, an initially unfair prediction can be remedied to, at least partially, meet a desired, weighted combination of the respective fairness conditions. We demonstrate the effectiveness of our algorithm when applied to synthetic data, the COMPAS data set, and a new, real-world data set from the e-commerce sector. Finally, we discuss to what extent FAIM can be harnessed to comply with conflicting legal obligations. The analysis suggests that it may operationalize duties in traditional legal fields, such as credit scoring and criminal justice proceedings, but also for the latest AI regulations put forth in the EU, like the Digital Markets Act and the recently enacted AI Act.