Differences in Performance of Bayesian Dynamic Borrowing and Synthetic Control Methods: A Case Study of Pediatric Atopic Dermatitis

TL;DR

This study compares Bayesian dynamic borrowing and synthetic control methods in clinical trials, focusing on power and error rates, using pediatric atopic dermatitis as a case study to inform method selection.

Contribution

It provides a direct comparison of BDB and SCM performance metrics in a real-world clinical trial context, highlighting their relative strengths and considerations.

Findings

SCM had higher power (0.641) than BDB (0.580) in the case study.

Both methods had similar low type 1 error rates (~0.026-0.027).

Choice of method should depend on specific trial needs.

Abstract

Bayesian dynamic borrowing (BDB) and synthetic control methods (SCM) are both used in clinical trial design when recruitment, retention, or allocation is a challenge. The performance of these approaches has not previously been directly compared due to differences in application, product, and measurement metrics. This study aims to conduct a comparison of power and type 1 error rates of BDB (using meta-analytic predictive prior (MAP)) and SCM using a case study of Pediatric Atopic Dermatitis. Six historical randomised control trials were selected for use in both the creation of the MAP prior and synthetic control arm. The R library RBesT was used to create a MAP prior and the R library Synthpop was used to create a synthetic control arm for the SCM. Power and type 1 error rate were used as comparison metrics. BDB produced a power of 0.580 and a type 1 error rate of 0.026. SCM produced a power of 0.641 and a type 1 error rate of 0.027. In this case study, the SCM model produced a higher power than the BDB method with a similar type 1 error rate. However, the decision to use SCM or BDB should come from the specific needs of the potential trial, since their power and type 1 error rate may differ on a case-by-case basis.