On boosting the power of Chatterjee's rank correlation

TL;DR

This paper introduces enhanced versions of Chatterjee's rank correlation that incorporate multiple nearest neighbors, significantly improving the efficiency of dependence testing, especially against Gaussian rotation alternatives.

Contribution

It proposes revised Chatterjee's rank correlations using multiple nearest neighbors, achieving near-parametric efficiency in dependence testing.

Findings

Achieves near-parametric efficiency in Gaussian rotation tests

Overcomes the rate-inefficiency of original Chatterjee's correlation

Maintains consistency in estimating the dependence measure

Abstract

Chatterjee (2021)'s ingenious approach to estimating a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the unusual property of being between 0 and 1, and being 0 or 1 if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely. However, more recent studies (Cao and Bickel, 2020; Shi et al., 2021b) showed that independence tests based on Chatterjee's rank correlation are unfortunately rate-inefficient against various local alternatives and they call for variants. We answer this call by proposing revised Chatterjee's rank correlations that still consistently estimate the same dependence measure but provably achieve near-parametric efficiency in testing against Gaussian rotation alternatives. This is possible via incorporating many right nearest neighbors in constructing the correlation coefficients. We thus overcome the "only one disadvantage" of Chatterjee's rank correlation (Chatterjee, 2021, Section 7).