On independence testing using the (partial) distance correlation

TL;DR

This paper evaluates the effectiveness of distance correlation and partial distance correlation in testing independence and conditional independence, comparing permutation and asymptotic p-value methods against classical Pearson correlation.

Contribution

It provides a comparative analysis of p-value computation methods for (conditional) independence tests using distance correlation, highlighting surprising findings especially for conditional independence.

Findings

Permutation and asymptotic methods show different performances.

Distance correlation detects non-linear dependencies.

Surprising results in conditional independence testing.

Abstract

Distance correlation is a measure of dependence between two paired random vectors or matrices of arbitrary, not necessarily equal, dimensions. Unlike Pearson correlation, the population distance correlation coefficient is zero if and only if the random vectors are independent. Thus, distance correlation measures both linear and non-linear association between two univariate and or multivariate random variables. Partial distance correlation expands to the case of conditional independence. To test for (conditional) independence, the p-value may be computed either via permutations or asymptotically via the $\chi^2$ distribution. In this paper we perform an intra-comparison of both approaches for (conditional) independence and an inter-comparison to the classical Pearson correlation where for the latter we compute the asymptotic p-value. The results are rather surprising, especially for the case of conditional independence.