Toward Scalable and Valid Conditional Independence Testing with Spectral Representations

TL;DR

This paper proposes a scalable, representation learning-based method for conditional independence testing that improves on existing kernel methods in validity, adaptivity, and scalability, with theoretical guarantees and promising preliminary results.

Contribution

It introduces a novel spectral representation approach for CI testing, combining SVD-based features with a contrastive learning algorithm, and provides theoretical analysis of its validity and power.

Findings

The method is theoretically valid and asymptotically powerful.

Preliminary experiments show improved scalability and performance.

The approach bridges kernel methods with modern representation learning.

Abstract

Conditional independence (CI) is central to causal inference, feature selection, and graphical modeling, yet it is untestable in many settings without additional assumptions. Existing CI tests often rely on restrictive structural conditions, limiting their validity on real-world data. Kernel methods using the partial covariance operator offer a more principled approach but suffer from limited adaptivity, slow convergence, and poor scalability. In this work, we explore whether representation learning can help address these limitations. Specifically, we focus on representations derived from the singular value decomposition of the partial covariance operator and use them to construct a simple test statistic, reminiscent of the Hilbert-Schmidt Independence Criterion (HSIC). We also introduce a practical bi-level contrastive algorithm to learn these representations. Our theory links representation learning error to test performance and establishes asymptotic validity and power guarantees. Preliminary experiments suggest that this approach offers a practical and statistically grounded path toward scalable CI testing, bridging kernel-based theory with modern representation learning.