"Detective Work We Shouldn't Have to Do": Practitioner Challenges in Regulatory-Aligned Data Quality in Machine Learning Systems

TL;DR

This study explores the challenges faced by EU-based data practitioners in ensuring regulatory-aligned data quality in ML systems, highlighting gaps, limitations, and needs for better tools and governance.

Contribution

It provides qualitative insights into practitioner challenges and needs regarding data quality compliance with EU regulations in ML systems.

Findings

Gaps between legal principles and engineering workflows

Fragmentation across data pipelines hampers compliance

Existing tools are limited and often reactive in quality management

Abstract

Ensuring data quality in machine learning (ML) systems has become increasingly complex as regulatory requirements expand. In the European Union (EU), frameworks such as the General Data Protection Regulation (GDPR) and the Artificial Intelligence Act (AI Act) articulate data quality requirements that closely parallel technical concerns in ML practice, while also extending to legal obligations related to accountability, risk management, and human rights protection. This paper presents a qualitative interview study with EU-based data practitioners working on ML systems in regulated contexts. Through semi-structured interviews, we investigate how practitioners interpret regulatory-aligned data quality, the challenges they encounter, and the supports they identify as necessary. Our findings reveal persistent gaps between legal principles and engineering workflows, fragmentation across data pipelines, limitations of existing tools, unclear responsibility boundaries between technical and legal teams, and a tendency toward reactive, audit-driven quality practices. We also identify practitioners' needs for compliance-aware tooling, clearer governance structures, and cultural shifts toward proactive data governance.