Point estimation for adaptive trial designs II: practical considerations and guidance

TL;DR

This paper discusses the impact of bias on point estimators in adaptive clinical trials, reviews current practices, and offers guidelines for choosing and reporting estimators to improve accuracy and reliability.

Contribution

It provides practical guidance on estimator selection and reporting in adaptive trials, emphasizing bias reduction and illustrating methods with real data and simulations.

Findings

Bias can significantly affect point estimates in adaptive trials.

Different estimators can yield noticeably different results in specific trial realizations.

Pre-specifying estimation strategies and using bias-reduced estimators improve estimate reliability.

Abstract

In adaptive clinical trials, the conventional end-of-trial point estimate of a treatment effect is prone to bias, that is, a systematic tendency to deviate from its true value. As stated in recent FDA guidance on adaptive designs, it is desirable to report estimates of treatment effects that reduce or remove this bias. However, it may be unclear which of the available estimators are preferable, and their use remains rare in practice. This paper is the second in a two-part series that studies the issue of bias in point estimation for adaptive trials. Part I provided a methodological review of approaches to remove or reduce the potential bias in point estimation for adaptive designs. In part II, we discuss how bias can affect standard estimators and assess the negative impact this can have. We review current practice for reporting point estimates and illustrate the computation of different estimators using a real adaptive trial example (including code), which we use as a basis for a simulation study. We show that while on average the values of these estimators can be similar, for a particular trial realisation they can give noticeably different values for the estimated treatment effect. Finally, we propose guidelines for researchers around the choice of estimators and the reporting of estimates following an adaptive design. The issue of bias should be considered throughout the whole lifecycle of an adaptive design, with the estimation strategy pre-specified in the statistical analysis plan. When available, unbiased or bias-reduced estimates are to be preferred.