High dimensional consistent independence testing with maxima of rank correlations

TL;DR

This paper introduces a new family of high-dimensional independence tests based on maxima of pairwise rank correlations, capable of detecting complex dependencies and proven to be rate-optimal under certain models.

Contribution

It develops a novel testing framework using maxima of rank correlations, supported by a new moderate deviation theorem, and establishes their optimality and distribution-free properties.

Findings

Tests are distribution-free for continuous margins.

Proposed tests are rate-optimal against sparse alternatives.

Revealed an identity between three rank correlation statistics.

Abstract

Testing mutual independence for high-dimensional observations is a fundamental statistical challenge. Popular tests based on linear and simple rank correlations are known to be incapable of detecting non-linear, non-monotone relationships, calling for methods that can account for such dependences. To address this challenge, we propose a family of tests that are constructed using maxima of pairwise rank correlations that permit consistent assessment of pairwise independence. Built upon a newly developed Cram\'{e}r-type moderate deviation theorem for degenerate U-statistics, our results cover a variety of rank correlations including Hoeffding's $D$, Blum-Kiefer-Rosenblatt's $R$, and Bergsma-Dassios-Yanagimoto's $\tau^*$. The proposed tests are distribution-free in the class of multivariate distributions with continuous margins, implementable without the need for permutation, and are shown to be rate-optimal against sparse alternatives under the Gaussian copula model. As a by-product of the study, we reveal an identity between the aforementioned three rank correlation statistics, and hence make a step towards proving a conjecture of Bergsma and Dassios.