Score-based Generative Modeling for Conditional Independence Testing

TL;DR

This paper introduces a new conditional independence testing method using score-based generative models, which improves accuracy and stability over previous GAN-based approaches, especially in high-dimensional data.

Contribution

The paper proposes a novel CI testing approach with score-based generative modeling, including a sliced conditional score matching scheme and Langevin dynamics sampling, with theoretical error bounds and validation.

Findings

Outperforms existing CI testing methods on synthetic datasets

Achieves precise Type I error control and high testing power

Demonstrates effectiveness on real-world datasets

Abstract

Determining conditional independence (CI) relationships between random variables is a fundamental yet challenging task in machine learning and statistics, especially in high-dimensional settings. Existing generative model-based CI testing methods, such as those utilizing generative adversarial networks (GANs), often struggle with undesirable modeling of conditional distributions and training instability, resulting in subpar performance. To address these issues, we propose a novel CI testing method via score-based generative modeling, which achieves precise Type I error control and strong testing power. Concretely, we first employ a sliced conditional score matching scheme to accurately estimate conditional score and use Langevin dynamics conditional sampling to generate null hypothesis samples, ensuring precise Type I error control. Then, we incorporate a goodness-of-fit stage into the method to verify generated samples and enhance interpretability in practice. We theoretically establish the error bound of conditional distributions modeled by score-based generative models and prove the validity of our CI tests. Extensive experiments on both synthetic and real-world datasets show that our method significantly outperforms existing state-of-the-art methods, providing a promising way to revitalize generative model-based CI testing.