Structured dataset documentation: a datasheet for CheXpert

TL;DR

This paper provides a detailed datasheet for the CheXpert dataset, emphasizing the importance of structured documentation for reliable medical image datasets to enhance machine learning applications in radiology.

Contribution

It introduces a comprehensive datasheet for CheXpert, illustrating how structured dataset documentation improves transparency and reliability in medical AI research.

Findings

Radiologist involvement ensures high-quality labels.

Structured documentation clarifies dataset composition and usage.

The datasheet serves as a model for dataset transparency.

Abstract

Billions of X-ray images are taken worldwide each year. Machine learning, and deep learning in particular, has shown potential to help radiologists triage and diagnose images. However, deep learning requires large datasets with reliable labels. The CheXpert dataset was created with the participation of board-certified radiologists, resulting in the strong ground truth needed to train deep learning networks. Following the structured format of Datasheets for Datasets, this paper expands on the original CheXpert paper and other sources to show the critical role played by radiologists in the creation of reliable labels and to describe the different aspects of the dataset composition in detail. Such structured documentation intends to increase the awareness in the machine learning and medical communities of the strengths, applications, and evolution of CheXpert, thereby advancing the field of medical image analysis. Another objective of this paper is to put forward this dataset datasheet as an example to the community of how to create detailed and structured descriptions of datasets. We believe that clearly documenting the creation process, the contents, and applications of datasets accelerates the creation of useful and reliable models.