It's complicated. The relationship of algorithmic fairness and non-discrimination provisions for high-risk systems in the EU AI Act

TL;DR

This paper examines how the EU AI Act integrates legal non-discrimination principles with algorithmic fairness, analyzing regulatory scope, challenges, and future interdisciplinary research needs.

Contribution

It provides an in-depth analysis of the relationship between EU non-discrimination law and AI fairness concepts, highlighting regulatory gaps and proposing future research directions.

Findings

Most non-discrimination regulations target high-risk AI systems

Regulations include data input and output monitoring, with some inconsistencies

Recommends developing specific auditing and testing methodologies for AI

Abstract

What constitutes a fair decision? This question is not only difficult for humans but becomes more challenging when Artificial Intelligence (AI) models are used. In light of discriminatory algorithmic behaviors, the EU has recently passed the AI Act, which mandates specific rules for high-risk systems, incorporating both traditional legal non-discrimination regulations and machine learning based algorithmic fairness concepts. This paper aims to bridge these two different concepts in the AI Act through: First, a necessary high-level introduction of both concepts targeting legal and computer science-oriented scholars, and second, an in-depth analysis of the AI Act's relationship between legal non-discrimination regulations and algorithmic fairness. Our analysis reveals three key findings: (1.) Most non-discrimination regulations target only high-risk AI systems. (2.) The regulation of high-risk systems encompasses both data input requirements and output monitoring, though these regulations are partly inconsistent and raise questions of computational feasibility. (3.) Finally, we consider the possible (future) interaction of classical EU non-discrimination law and the AI Act regulations. We recommend developing more specific auditing and testing methodologies for AI systems. This paper aims to serve as a foundation for future interdisciplinary collaboration between legal scholars and computer science-oriented machine learning researchers studying discrimination in AI systems.