The asymptotic distribution and Berry--Esseen bound of a new test for independence in high dimension with an application to stochastic optimization

TL;DR

This paper introduces a new high-dimensional independence test statistic, establishes its asymptotic distribution with a fast convergence rate, and demonstrates its effectiveness through simulations and applications in stochastic optimization.

Contribution

The paper proposes a novel independence test statistic for high-dimensional data and derives its asymptotic distribution with an improved convergence rate over existing methods.

Findings

The test statistic's limiting distribution is an extreme distribution of type I.

Convergence rate of the distribution approximation is $O(( ext{log } n)^{5/2}/ ext{sqrt } n)$.

Simulation results confirm the theoretical properties and practical utility of the test.

Abstract

Let $\mathbf{X}_1,...,\mathbf{X}_n$ be a random sample from a $p$-dimensional population distribution. Assume that $c_1n^{\alpha}\leq p\leq c_2n^{\alpha}$ for some positive constants $c_1,c_2$ and $\alpha$. In this paper we introduce a new statistic for testing independence of the $p$-variates of the population and prove that the limiting distribution is the extreme distribution of type I with a rate of convergence $O((\log n)^{5/2}/\sqrt{n})$. This is much faster than $O(1/\log n)$, a typical convergence rate for this type of extreme distribution. A simulation study and application to stochastic optimization are discussed.