Combining Observational and Experimental Data to Improve Efficiency Using Imperfect Instruments

TL;DR

This paper introduces a method that combines observational and experimental data using imperfect instruments to significantly enhance estimation efficiency, reducing required sample sizes by up to 50%.

Contribution

It develops a novel approach leveraging imperfect instruments to improve causal effect estimation by combining observational and experimental data.

Findings

Variance can be reduced by up to 50%.

Method improves precision in real-world dataset.

Requires only half the experimental sample size.

Abstract

Randomized controlled trials generate experimental variation that can credibly identify causal effects, but often suffer from limited scale, while observational datasets are large, but often violate desired identification assumptions. To improve estimation efficiency, I propose a method that leverages imperfect instruments - pretreatment covariates that satisfy the relevance condition but may violate the exclusion restriction. I show that these imperfect instruments can be used to derive moment restrictions that, in combination with the experimental data, improve estimation efficiency. I outline estimators for implementing this strategy, and show that my methods can reduce variance by up to 50%; therefore, only half of the experimental sample is required to attain the same statistical precision. I apply my method to a search listing dataset from Expedia that studies the causal effect of search rankings on clicks, and show that the method can substantially improve the precision.