Sensitivity Analysis for matched pair analysis of binary data: From worst case to average case analysis

TL;DR

This paper proposes a shift from worst-case to average-case bias calibration in sensitivity analysis for matched pair studies with binary data, leading to less conservative and more powerful assessments of treatment effects.

Contribution

It introduces an average bias calibration method for sensitivity analysis, providing a more natural and potentially less conservative alternative to the traditional worst-case approach.

Findings

Average bias calibration yields less conservative sensitivity analysis.

The method improves power in detecting treatment effects.

Application to cellphone use and automobile accidents demonstrates practical utility.

Abstract

In matched observational studies where treatment assignment is not randomized, sensitivity analysis helps investigators determine how sensitive their estimated treatment effect is to some unmeasured con- founder. The standard approach calibrates the sensitivity analysis according to the worst case bias in a pair. This approach will result in a conservative sensitivity analysis if the worst case bias does not hold in every pair. In this paper, we show that for binary data, the standard approach can be calibrated in terms of the average bias in a pair rather than worst case bias. When the worst case bias and average bias differ, the average bias interpretation results in a less conservative sensitivity analysis and more power. In many studies, the average case calibration may also carry a more natural interpretation than the worst case calibration and may also allow researchers to incorporate additional data to establish an empirical basis with which to calibrate a sensitivity analysis. We illustrate this with a study of the effects of cellphone use on the incidence of automobile accidents. Finally, we extend the average case calibration to the sensitivity analysis of confidence intervals for attributable effects.