Bias by Design? How Data Practices Shape Fairness in AI Healthcare Systems

TL;DR

This paper examines how biased data collection practices in healthcare AI systems lead to fairness issues, identifying various bias types and offering practical recommendations to improve data quality and fairness in clinical AI applications.

Contribution

It provides a detailed analysis of biases in clinical data collection and proposes strategies to enhance fairness and robustness in healthcare AI systems.

Findings

Identified multiple bias types including historical, representation, and measurement biases.

Biases are present in variables like sex, age, socioeconomic status, and equipment.

Practical recommendations for fairer data collection practices.

Abstract

Artificial intelligence (AI) holds great promise for transforming healthcare. However, despite significant advances, the integration of AI solutions into real-world clinical practice remains limited. A major barrier is the quality and fairness of training data, which is often compromised by biased data collection practices. This paper draws on insights from the AI4HealthyAging project, part of Spain's national R&D initiative, where our task was to detect biases during clinical data collection. We identify several types of bias across multiple use cases, including historical, representation, and measurement biases. These biases manifest in variables such as sex, gender, age, habitat, socioeconomic status, equipment, and labeling. We conclude with practical recommendations for improving the fairness and robustness of clinical problem design and data collection. We hope that our findings and experience contribute to guiding future projects in the development of fairer AI systems in healthcare.