Analysis of Conditional Randomisation and Permutation schemes with application to conditional independence testing

TL;DR

This paper analyzes the properties of Conditional Randomisation and Permutation schemes for testing conditional independence, deriving their asymptotic behaviors, validating permutation p-values, and comparing their performance in statistical tests.

Contribution

It provides a theoretical framework for understanding the asymptotic distributions of estimates in resampling schemes and validates the use of adjusted chi-square tests for conditional independence testing.

Findings

Asymptotic normality of probability estimates established

Distributions of empirical Conditional Mutual Information coincide for both schemes

Permutation p-values are valid for the Conditional Permutation scheme

Abstract

We study properties of two resampling scenarios: Conditional Randomisation and Conditional Permutation schemes, which are relevant for testing conditional independence of discrete random variables $X$ and $Y$ given a random variable $Z$. Namely, we investigate asymptotic behaviour of estimates of a vector of probabilities in such settings, establish their asymptotic normality and ordering between asymptotic covariance matrices. The results are used to derive asymptotic distributions of the empirical Conditional Mutual Information in those set-ups. Somewhat unexpectedly, the distributions coincide for the two scenarios, despite differences in the asymptotic distributions of the estimates of probabilities. We also prove validity of permutation p-values for the Conditional Permutation scheme. The above results justify consideration of conditional independence tests based on resampled p-values and on the asymptotic chi-square distribution with an adjusted number of degrees of freedom. We show in numerical experiments that when the ratio of the sample size to the number of possible values of the triple exceeds 0.5, the test based on the asymptotic distribution with the adjustment made on a limited number of permutations is a viable alternative to the exact test for both the Conditional Permutation and the Conditional Randomisation scenarios. Moreover, there is no significant difference between the performance of exact tests for Conditional Permutation and Randomisation schemes, the latter requiring knowledge of conditional distribution of $X$ given $Z$, and the same conclusion is true for both adaptive tests.