Sensitivity Analyses of Clinical Trial Designs: Selecting Scenarios and Summarizing Operating Characteristics

TL;DR

This paper introduces a new optimization-based method for selecting representative simulation scenarios in sensitivity analyses of clinical trial designs, improving the assessment of how design characteristics depend on unknown parameters.

Contribution

It proposes a formal approach to choose and optimize the set of scenarios for sensitivity analyses, balancing interpretability and comprehensive coverage of plausible parameter values.

Findings

New scenario selection method minimizes loss function for adequacy

Supports optimal number of scenarios for sensitivity analysis

Enhances understanding of design operating characteristics

Abstract

The use of simulation-based sensitivity analyses is fundamental to evaluate and compare candidate designs for future clinical trials. In this context, sensitivity analyses are especially useful to assess the dependence of important design operating characteristics (OCs) with respect to various unknown parameters (UPs). Typical examples of OCs include the likelihood of detecting treatment effects and the average study duration, which depend on UPs that are not known until after the onset of the clinical study, such as the distributions of the primary outcomes and patient profiles. Two crucial components of sensitivity analyses are (i) the choice of a set of plausible simulation scenarios $\{\boldsymbol{\theta}_1,...,\boldsymbol{\theta}_K\}$ and (ii) the list of OCs of interest. We propose a new approach to choose the set of scenarios for inclusion in design sensitivity analyses. Our approach balances the need for simplicity and interpretability of OCs computed across several scenarios with the need to faithfully summarize -- through simulations -- how the OCs vary across all plausible values of the UPs. Our proposal also supports the selection of the number of simulation scenarios to be included in the final sensitivity analysis report. To achieve these goals, we minimize a loss function $\mathcal{L}(\boldsymbol{\theta}_1,...,\boldsymbol{\theta}_K)$ that formalizes whether a specific set of $K$ sensitivity scenarios $\{\boldsymbol{\theta}_1,...,\boldsymbol{\theta}_K\}$ is adequate to summarize how the OCs of the trial design vary across all plausible values of the UPs. Then, we use optimization techniques to select the best set of simulation scenarios to exemplify the OCs of the trial design.