Statistical Classification via Robust Hypothesis Testing: Non-Asymptotic and Simple Bounds

TL;DR

This paper develops simple, non-asymptotic bounds for robust Bayesian classification using hypothesis testing, showing how training data size and alphabet size affect error rates, with practical implications for large alphabet sources.

Contribution

It introduces non-asymptotic exponential bounds for robust hypothesis testing in Bayesian classification, applicable to large alphabets and providing insights into training data effects.

Findings

Error bounds depend on training sequence length and alphabet size.

Performance approaches optimal as training data increases.

Method applicable to large alphabet sources with sub-quadratic growth.

Abstract

We consider Bayesian multiple statistical classification problem in the case where the unknown source distributions are estimated from the labeled training sequences, then the estimates are used as nominal distributions in a robust hypothesis test. Specifically, we employ the DGL test due to Devroye et al. and provide non-asymptotic, exponential upper bounds on the error probability of classification. The proposed upper bounds are simple to evaluate and reveal the effects of the length of the training sequences, the alphabet size and the numbers of hypothesis on the error exponent. The proposed method can also be used for large alphabet sources when the alphabet grows sub-quadratically in the length of the test sequence. The simulations indicate that the performance of the proposed method gets close to that of optimal hypothesis testing as the length of the training sequences increases.