Sensitivity Analysis for Unmeasured Confounding in Medical Product Development and Evaluation Using Real World Evidence

TL;DR

This paper reviews sensitivity analysis methods for unmeasured confounding in non-randomized real-world evidence studies, emphasizing practical interpretation and regulatory considerations to improve medical product evaluation.

Contribution

It provides a comprehensive review of sensitivity analysis techniques for unmeasured confounding in RWE studies and demonstrates their application in regulatory decision-making.

Findings

Sensitivity analysis methods vary by confounding type

Practical interpretation aids regulatory decisions

Case study illustrates real-world application

Abstract

The American Statistical Association Biopharmaceutical Section (ASA BIOP) scientific working group on real-world evidence (RWE) has been making continuous, extended efforts towards a goal of supporting and advancing regulatory science with respect to clinical studies intended to use real-world data for evidence generation for the purpose of medical product development and evaluation (i.e., RWE studies). In 2023, the working group published a manuscript delineating challenges and opportunities in constructing estimands for RWE studies following the framework in ICH E9(R1) guidance on estimand and sensitivity analysis. As a follow-up task, we describe the other issue, sensitivity analysis. Although the FDA's definition of RWE studies includes randomized trials using RWD as a primary source of evidence generation such as pragmatic trials, here we focus on non-randomized RWE studies and the issue of unmeasured confounding which is a major source of bias for most RWE studies. We review the availability and applicability of sensitivity analysis methods for different types of unmeasured confounding. We also provide some practical and regulatory considerations on using sensitivity analysis for such RWE studies. Then, we use a plasmode case study to demonstrate realistic ways to interpret sensitivity analysis results in supporting regulatory decision-making. We conclude with a brief discussion.