Nonparametric Composite Hypothesis Testing in an Asymptotic Regime

TL;DR

This paper introduces the concept of discrimination capacity for nonparametric composite hypothesis testing with infinitely many hypotheses, analyzing error exponents using distributional distances like MMD and KS, and providing bounds and numerical validation.

Contribution

It defines discrimination capacity in an asymptotic regime and characterizes it for nonparametric tests using MMD and KS distances, including bounds and validation.

Findings

Discrimination capacity quantifies the exponential growth of hypotheses with vanishing error.

Lower bounds on discrimination capacity are derived for MMD and KS-based tests.

An upper bound based on Fano's inequality is established.

Abstract

We investigate the nonparametric, composite hypothesis testing problem for arbitrary unknown distributions in the asymptotic regime where both the sample size and the number of hypotheses grow exponentially large. Such asymptotic analysis is important in many practical problems, where the number of variations that can exist within a family of distributions can be countably infinite. We introduce the notion of \emph{discrimination capacity}, which captures the largest exponential growth rate of the number of hypotheses relative to the sample size so that there exists a test with asymptotically vanishing probability of error. Our approach is based on various distributional distance metrics in order to incorporate the generative model of the data. We provide analyses of the error exponent using the maximum mean discrepancy (MMD) and Kolmogorov-Smirnov (KS) distance and characterize the corresponding discrimination rates, i.e., lower bounds on the discrimination capacity, for these tests. Finally, an upper bound on the discrimination capacity based on Fano's inequality is developed. Numerical results are presented to validate the theoretical results.