Trust Me, I'm a Doctor?

TL;DR

This paper explores how combined randomized and observational data can evaluate whether physicians outperform standard trial-based treatment strategies for individual patients.

Contribution

It introduces a formal framework with bounds to assess when physician discretion surpasses trial-based recommendations using nested data.

Findings

Derived sharp bounds on physician performance relative to trial strategies.

Identified conditions where clinical data justify physician discretion over trial recommendations.

Provided insights into when stronger evidence is needed for personalized treatment decisions.

Abstract

Clinical trials usually target average treatment effects, but treatment decisions are made for individuals. This tension motivates a common criticism of evidence-based medicine: a treatment that is beneficial on average may be inappropriate for a particular patient, and skilled physicians may outperform rigid adherence to the strategy that performed best in a randomized trial. We consider how randomized and observational data from the same target population can be used to assess that possibility. Specifically, we study settings in which a randomized trial is nested within an observational cohort, so that outcomes are observed under treatment, control, and usual care. We ask what the observed data can reveal about how often physicians outperform the strategy suggested by the trial. We define a gain score to formalize this comparison and derive sharp bounds on the proportion of physicians whose personal strategies perform at least as well as, or better than, always choosing the better performing treatment from the trial. These results shed light on when clinical data support relying on physician discretion over the trial-average recommendation and when stronger justification is required.