Amortized Conditional Independence Testing

TL;DR

This paper introduces ACID, a transformer-based neural network that learns to test for conditional independence, enabling fast, data-driven, and adaptable causal discovery with state-of-the-art performance.

Contribution

The paper presents ACID, a novel amortized conditional independence testing method using transformers, which can be trained on synthetic data and applied across various datasets efficiently.

Findings

ACID outperforms existing methods on synthetic and real data benchmarks.

ACID generalizes well to different sample sizes and non-linearities.

ACID achieves low inference time and high robustness.

Abstract

Testing for the conditional independence structure in data is a fundamental and critical task in statistics and machine learning, which finds natural applications in causal discovery - a highly relevant problem to many scientific disciplines. Existing methods seek to design explicit test statistics that quantify the degree of conditional dependence, which is highly challenging yet cannot capture nor utilize prior knowledge in a data-driven manner. In this study, an entirely new approach is introduced, where we instead propose to amortize conditional independence testing and devise ACID - a novel transformer-based neural network architecture that learns to test for conditional independence. ACID can be trained on synthetic data in a supervised learning fashion, and the learned model can then be applied to any dataset of similar natures or adapted to new domains by fine-tuning with a negligible computational cost. Our extensive empirical evaluations on both synthetic and real data reveal that ACID consistently achieves state-of-the-art performance against existing baselines under multiple metrics, and is able to generalize robustly to unseen sample sizes, dimensionalities, as well as non-linearities with a remarkably low inference time.