Combining Observational and Experimental Datasets Using Shrinkage Estimators

TL;DR

This paper introduces new shrinkage estimators for combining observational and experimental data to improve causal inference, especially when unconfoundedness cannot be assumed, using risk estimation and sensitivity analysis.

Contribution

It proposes a generic shrinkage estimator framework, develops two novel estimators with risk guarantees, and connects the approach to sensitivity analysis methods.

Findings

New estimators outperform purely experimental data under certain conditions

Establishes finite sample risk bounds for the proposed estimators

Provides a method to assess the feasibility of the estimators in practice

Abstract

We consider the problem of combining data from observational and experimental sources to make causal conclusions. This problem is increasingly relevant, as the modern era has yielded passive collection of massive observational datasets in areas such as e-commerce and electronic health. These data may be used to supplement experimental data, which is frequently expensive to obtain. In Rosenman et al. (2018), we considered this problem under the assumption that all confounders were measured. Here, we relax the assumption of unconfoundedness. To derive combined estimators with desirable properties, we make use of results from the Stein Shrinkage literature. Our contributions are threefold. First, we propose a generic procedure for deriving shrinkage estimators in this setting, making use of a generalized unbiased risk estimate. Second, we develop two new estimators, prove finite sample conditions under which they have lower risk than an estimator using only experimental data, and show that each achieves a notion of asymptotic optimality. Third, we draw connections between our approach and results in sensitivity analysis, including proposing a method for evaluating the feasibility of our estimators.