Universal Neyman-Pearson Classification with a Known Hypothesis

TL;DR

This paper introduces a universal binary classifier for Neyman-Pearson testing with a known null distribution and limited alternative data, achieving near-optimal error bounds and extending to sequential testing.

Contribution

It develops a classifier that interpolates between Hoeffding's test and the likelihood ratio test, with theoretical guarantees on error probabilities and exponents.

Findings

Classifier attains the same error probability prefactor as the likelihood ratio test.

Achieves the optimal error exponent tradeoff under certain training-to-observation ratios.

Proposes a sequential classifier that maintains optimal error exponent tradeoff.

Abstract

We propose a universal classifier for binary Neyman-Pearson classification where null distribution is known while only a training sequence is available for the alternative distribution. The proposed classifier interpolates between Hoeffding's classifier and the likelihood ratio test and attains the same error probability prefactor as the likelihood ratio test, i.e., the same prefactor as if both distributions were known. In addition, like Hoeffding's universal hypothesis test, the proposed classifier is shown to attain the optimal error exponent tradeoff attained by the likelihood ratio test whenever the ratio of training to observation samples exceeds a certain value. We propose a lower bound and an upper bound to the training to observation ratio. In addition, we propose a sequential classifier that attains the optimal error exponent tradeoff.