# Will the EU AI Act help to mitigate dataset bias in medical AI?

## TL;DR

This paper examines how the EU AI Act could help reduce bias in medical AI by setting rules for dataset quality and governance.

## Contribution

The paper analyzes the EU AI Act's provisions on dataset bias mitigation and their potential impact on medical AI systems.

## Key findings

- The AI Act introduces requirements for dataset representativeness and contextual considerations to mitigate bias.
- The 'debiasing exception' allows processing of sensitive data under certain conditions to address bias.
- Enforcement mechanisms include granting authorities access to datasets, though data protection challenges remain.

## Abstract

The aim of this article is to provide an overview and analyze the implications of the provisions on dataset quality and bias in the AI Act (AIA). The AIA requires providers of AI systems to take measures to identify, prevent, and mitigate biases as part of the data governance practices. The AIA also explicitly prescribes certain characteristics required of training, validation, and testing datasets. These include notions widely considered as best practice such as representativeness as well as consideration of characteristics particular to the “geographical, contextual, behavioural or functional setting” which might expand the scope of considerations already common among AI developers. The AIA also aims to address the legal limitations on access to sensitive data by introducing the so called “debiasing exception,” which under certain conditions permits the processing of sensitive data for debiasing purposes. To ensure enforcement of the data governance provisions, the AIA grants notified bodies and enforcement authorities access to training, validation, and testing datasets; however, further efforts may be needed to reconcile data protection concerns with these enforcement powers. The AIA’s requirements will likely help mitigate bias in medical AI systems. Associated soft law instruments should contribute to the effective implementation of these requirements.