Long-Term Effect Estimation with Surrogate Representation

TL;DR

This paper introduces a novel framework for estimating long-term causal effects using surrogate representations, addressing confounding bias and surrogacy assumptions by linking long-term causal inference with sequential machine learning models.

Contribution

It proposes a new method that learns surrogate representations to improve long-term effect estimation without relying on strong surrogacy assumptions.

Findings

Outperforms state-of-the-art methods in experiments

Effectively handles confounding bias in long-term causal inference

Circumvents strict surrogacy assumptions using temporal models

Abstract

There are many scenarios where short- and long-term causal effects of an intervention are different. For example, low-quality ads may increase short-term ad clicks but decrease the long-term revenue via reduced clicks. This work, therefore, studies the problem of long-term effect where the outcome of primary interest, or primary outcome, takes months or even years to accumulate. The observational study of long-term effect presents unique challenges. First, the confounding bias causes large estimation error and variance, which can further accumulate towards the prediction of primary outcomes. Second, short-term outcomes are often directly used as the proxy of the primary outcome, i.e., the surrogate. Nevertheless, this method entails the strong surrogacy assumption that is often impractical. To tackle these challenges, we propose to build connections between long-term causal inference and sequential models in machine learning. This enables us to learn surrogate representations that account for the temporal unconfoundedness and circumvent the stringent surrogacy assumption by conditioning on the inferred time-varying confounders. Experimental results show that the proposed framework outperforms the state-of-the-art.