Detecting non-uniform patterns on high-dimensional hyperspheres

TL;DR

This paper introduces a new probabilistic method and test for detecting non-uniform patterns on high-dimensional hyperspheres, achieving minimax optimality and consistency across various models.

Contribution

It develops a novel distance-based test for spherical uniformity that is theoretically optimal and universally consistent in high-dimensional settings.

Findings

The proposed test is minimax-optimal over several high-dimensional parametric models.

It is consistent against non-local high-dimensional alternatives.

The test's asymptotic distribution under alternatives is characterized.

Abstract

We propose a new probabilistic characterization of the uniform distribution on the hypersphere in terms of the distribution of pairwise inner products, extending the ideas of \citep{cuesta2009projection,cuesta2007sharp} in a data-driven manner. This characterization naturally leads to an Ingster-type distance for quantifying deviations from uniformity, whose asymptotic behavior can be analyzed systematically via Edgeworth-type expansions. Perhaps surprisingly, we show that this distance captures the minimax rates for testing uniformity simultaneously across several high-dimensional parametric models, even in the models where densities with respect to the uniform law do not exist. We then introduce a simple test for spherical uniformity based on this distance and study its detection rates and consistency against various classes of alternatives, both local and non-local. The proposed test is universally consistent in fixed dimensions, minimax-optimal over a variety of high-dimensional parametric models, and consistent against non-local high-dimensional alternatives. This is different from previously studied high-dimensional Sobolev tests and extreme-value-based tests, which are rate-suboptimal or inconsistent against one or more classes of alternatives. We also establish the local asymptotic distribution of the proposed test under the considered classes of alternatives, along with new information lower bounds.