Relative Bias Under Imperfect Identification in Observational Causal Inference

TL;DR

This paper examines how violations of assumptions affect bias in observational causal inference methods and introduces sensitivity tools to compare strategies under assumption violations.

Contribution

It develops bias formulas for IV and proximal inference under assumption violations and proposes sensitivity analysis tools and visualizations for better strategy comparison.

Findings

Bias increases with unmeasured confounding in IV and proximal methods.

Sensitivity analysis helps understand the impact of assumption violations.

Re-analysis of surveillance and protest data illustrates the methods.

Abstract

To conduct causal inference in observational settings, researchers must rely on certain identifying assumptions. In practice, these assumptions are unlikely to hold exactly. This paper considers the bias of selection-on-observables, instrumental variables, and proximal inference estimates under violations of their identifying assumptions. We develop bias expressions for IV and proximal inference that show how violations of their respective assumptions are amplified by any unmeasured confounding in the outcome variable. We propose a set of sensitivity tools that quantify the sensitivity of different identification strategies, and an augmented bias contour plot visualizes the relationship between these strategies. We argue that the act of choosing an identification strategy implicitly expresses a belief about the degree of violations that must be present in alternative identification strategies. Even when researchers intend to conduct an IV or proximal analysis, a sensitivity analysis comparing different identification strategies can help to better understand the implications of each set of assumptions. Throughout, we compare the different approaches on a re-analysis of the impact of state surveillance on the incidence of protest in Communist Poland.