Estimating long-term causal effects from short-term experiments and long-term observational data with unobserved confounding

TL;DR

This paper develops a method to estimate long-term causal effects by combining short-term experimental data with long-term observational data, addressing unobserved confounding through an innovative instrumental variable approach.

Contribution

It introduces a novel estimator that accounts for unobserved confounders in long-term causal effect estimation using combined data sources.

Findings

Estimator is proven unbiased.

Analytical variance study conducted.

Validated on synthetic and real stroke trial data.

Abstract

Understanding and quantifying cause and effect is an important problem in many domains. The generally-agreed solution to this problem is to perform a randomised controlled trial. However, even when randomised controlled trials can be performed, they usually have relatively short duration's due to cost considerations. This makes learning long-term causal effects a very challenging task in practice, since the long-term outcome is only observed after a long delay. In this paper, we study the identification and estimation of long-term treatment effects when both experimental and observational data are available. Previous work provided an estimation strategy to determine long-term causal effects from such data regimes. However, this strategy only works if one assumes there are no unobserved confounders in the observational data. In this paper, we specifically address the challenging case where unmeasured confounders are present in the observational data. Our long-term causal effect estimator is obtained by combining regression residuals with short-term experimental outcomes in a specific manner to create an instrumental variable, which is then used to quantify the long-term causal effect through instrumental variable regression. We prove this estimator is unbiased, and analytically study its variance. In the context of the front-door causal structure, this provides a new causal estimator, which may be of independent interest. Finally, we empirically test our approach on synthetic-data, as well as real-data from the International Stroke Trial.