A Robustness Test for Estimating Total Effects with Covariate Adjustment

TL;DR

This paper introduces a robustness testing method for estimating total effects in causal models, leveraging multiple valid adjustment sets to verify the correctness of the causal graph used.

Contribution

It proposes a novel testing procedure that exploits multiple valid adjustment sets to assess the reliability of the causal graph in estimating total effects.

Findings

The test can detect certain errors in the causal graph.

It provides a robustness check for covariate adjustment.

Connections to econometrics stability tests are established.

Abstract

Suppose we want to estimate a total effect with covariate adjustment in a linear structural equation model. We have a causal graph to decide what covariates to adjust for, but are uncertain about the graph. Here, we propose a testing procedure, that exploits the fact that there are multiple valid adjustment sets for the target total effect in the causal graph, to perform a robustness check on the graph. If the test rejects, it is a strong indication that we should not rely on the graph. We discuss what mistakes in the graph our testing procedure can detect and which ones it cannot and develop two strategies on how to select a list of valid adjustment sets for the procedure. We also connect our result to the related econometrics literature on coefficient stability tests.