Fisher's Randomization Test for Causality with General Types of Treatments

TL;DR

This paper extends Fisher's randomization test to assess conditional independence between outcomes and treatments with any variable type, under a generalized unconfoundedness assumption, applicable to both randomized and observational studies.

Contribution

It introduces a flexible, assumption-light Fisher's test framework for causal inference that handles diverse treatment types and includes a novel sensitivity analysis for unobserved confounding.

Findings

Valid Type I error control under generalized unconfoundedness

Construction of optimal test statistics from Bayes factors

Application to observational data demonstrating robustness

Abstract

We extend Fisher's randomization test (FRT) to test conditional independence between observed outcomes and treatments given covariates in both randomized experiments and observational studies, with no restriction on the variable type of treatments. Under a generalized unconfoundedness assumption, we provide causal identification for this hypothesis. Our approach requires neither the no-interference nor the positive overlap assumption, making it a widely applicable tool for detecting causal effects. A unique advantage of FRT lies in the separated roles of assignment and outcome models. The former, whether known from randomized experiments or estimated in observational studies, guarantees valid Type I error control at least asymptotically. The latter, even if misspecified, is used to construct optimal test statistics derived from Bayes factors. The synthesis of two classes of models through FRT yields a calibrated Bayesian procedure with desired frequentist properties. Recognizing that the generalized unconfoundedness assumption is untestable in observational studies, we develop a novel sensitivity analysis to assess the robustness of causal conclusions to unobserved confounding. Through a re-analysis of a panel dataset, we show how our methods can be integrated into a pipeline for observational causal inference.