Falsification of Internal and External Validity in Observational Studies via Conditional Moment Restrictions

TL;DR

This paper introduces a method to validate causal effect estimates from observational studies using RCT data by testing conditional moment restrictions, improving reliability and interpretability of causal inferences.

Contribution

It develops a test based on Conditional Moment Restrictions that assesses internal and external validity of observational causal estimates using RCT data, with proven asymptotic guarantees.

Findings

The proposed test has superior power and controlled type I error.

The method is interpretable and allows visualization of subgroups causing falsification.

Demonstrated effectiveness on semi-synthetic and real datasets.

Abstract

Randomized Controlled Trials (RCT)s are relied upon to assess new treatments, but suffer from limited power to guide personalized treatment decisions. On the other hand, observational (i.e., non-experimental) studies have large and diverse populations, but are prone to various biases (e.g. residual confounding). To safely leverage the strengths of observational studies, we focus on the problem of falsification, whereby RCTs are used to validate causal effect estimates learned from observational data. In particular, we show that, given data from both an RCT and an observational study, assumptions on internal and external validity have an observable, testable implication in the form of a set of Conditional Moment Restrictions (CMRs). Further, we show that expressing these CMRs with respect to the causal effect, or "causal contrast", as opposed to individual counterfactual means, provides a more reliable falsification test. In addition to giving guarantees on the asymptotic properties of our test, we demonstrate superior power and type I error of our approach on semi-synthetic and real world datasets. Our approach is interpretable, allowing a practitioner to visualize which subgroups in the population lead to falsification of an observational study.