Scorecards for Synthetic Medical Data Evaluation and Reporting

TL;DR

This paper introduces a standardized evaluation framework and a reporting tool called SMD Card for assessing the quality of synthetic medical data, aiming to facilitate its adoption in AI development and regulatory processes.

Contribution

It presents a novel comprehensive evaluation framework and reporting standard specifically designed for synthetic medical data, addressing current gaps in quality assessment.

Findings

Development of the SMD Card reporting tool

Framework tailored to medical data requirements

Enhances transparency and regulatory compliance

Abstract

Although interest in synthetic medical data (SMD) for training and testing AI methods is growing, the absence of a standardized framework to evaluate its quality and applicability hinders its wider adoption. Here, we outline an evaluation framework designed to meet the unique requirements of medical applications, and introduce SMD Card, which can serve as comprehensive reports that accompany artificially generated datasets. This card provides a transparent and standardized framework for evaluating and reporting the quality of synthetic data, which can benefit SMD developers, users, and regulators, particularly for AI models using SMD in regulatory submissions.