Biomedical Data Manifest: A lightweight data documentation mapping to increase transparency for AI/ML

TL;DR

The paper introduces a new data documentation framework called the Biomedical Data Manifest to improve transparency and reduce bias in biomedical ML datasets.

Contribution

The novel contribution is a modular, role-specific documentation template that reduces generator burden while ensuring relevant information for end-users.

Findings

A two-step process identified key documentation fields and role-specific priorities among biomedical stakeholders.

The Biomedical Data Manifest was developed to provide modular and transparent dataset documentation.

The framework supports transparency and bias mitigation in datasets used for ML applications.

Abstract

Biomedical machine learning (ML) models raise critical concerns about embedded assumptions influencing clinical decision-making, necessitating robust documentation frameworks for datasets that are shared via external repositories. Fairness-aware algorithm effectiveness hinges on users’ prior awareness of specific issues in the data – information such as data collection methodology, provenance and quality. Current ML-focused documentation approaches impose impractical burdens on data generators and conflate data/model accountability. This is problematic for resource datasets not explicitly created for ML applications. This study addresses these gaps through a two-step process: First, we derived consensus documentation fields by mapping elements across four key templates. Second, we surveyed biomedical stakeholders across four roles (clinicians, bench scientists, data manager and computationalists) to assess field importance and relevance. This revealed important role-dependent prioritization differences, motivating the development of the Biomedical Data Manifest – a modular template employing persona-specific field presentation reducing generator burden while ensuring end-users receive role-relevant information. The Biomedical Data Manifest improves transparency for datasets deposited in public or controlled-access repositories and bias mitigation across ML applications.

Figures