Using sensitive data to de-bias AI systems: Article 10(5) of the EU AI Act

TL;DR

This paper analyzes the EU AI Act's new provision allowing the collection of sensitive data to evaluate and mitigate biases in AI datasets, aiming to prevent discrimination while complying with legal obligations.

Contribution

It clarifies the scope and legal implications of Article 10(5) of the EU AI Act regarding sensitive data collection for bias mitigation.

Findings

Provides legal analysis of Article 10(5) AI Act

Highlights potential benefits for bias reduction

Discusses international relevance of the regulation

Abstract

In June 2024, the EU AI Act came into force. The AI Act includes obligations for the provider of an AI system. Article 10 of the AI Act includes a new obligation for providers to evaluate whether their training, validation and testing datasets meet certain quality criteria, including an appropriate examination of biases in the datasets and correction measures. With the obligation comes a new provision in Article 10(5) AI Act, allowing providers to collect sensitive data to fulfil the obligation. The exception aims to prevent discrimination. In this paper, I research the scope and implications of Article 10(5) AI Act. The paper primarily concerns European Union law, but may be relevant in other parts of the world, as policymakers aim to regulate biases in AI systems.