Deep Autoregressive Models as Causal Inference Engines

TL;DR

This paper introduces an autoregressive framework for causal inference that handles complex confounders and sequential actions, enabling direct probability computation and improved predictions in complex scenarios.

Contribution

It proposes a novel autoregressive causal inference method using sequencification, capable of modeling complex data and estimating multiple causal effects with a single model.

Findings

Effective in maze navigation, chess endgames, and keyword impact analysis.

Handles complex confounders and sequential actions.

Enables direct computation of interventional probabilities.

Abstract

Existing causal inference (CI) models are often restricted to data with low-dimensional confounders and singleton actions. We propose an autoregressive (AR) CI framework capable of handling complex confounders and sequential actions commonly found in modern applications. Our approach accomplishes this using {\em sequencification}, which transforms data from an underlying causal diagram into a sequence of tokens. Sequencification not only accommodates training with data generated from a large class of DAGs, but also extends existing CI capabilities to estimate multiple causal quantities using a {\em single} model. We can directly compute probabilities from interventional distributions, simplifying inference and improving outcome prediction accuracy. We demonstrate that an AR model adapted for CI is efficient and effective in various complex applications such as navigating mazes, playing chess endgames, and evaluating the impact of certain keywords on paper acceptance rates, where we consider causal queries beyond standard reinforcement learning-type questions.