Finite-Length Bounds on Hypothesis Testing Subject to Vanishing Type I Error Restrictions

TL;DR

This paper derives finite-length bounds on the tradeoff between Type I and Type II errors in binary hypothesis testing, extending previous asymptotic results to finite sample scenarios using concentration inequalities.

Contribution

It introduces new non-asymptotic bounds for hypothesis testing with vanishing Type I error restrictions, extending Strassen's 2009 results to finite sample sizes.

Findings

Derived upper and lower bounds for Type II error probability

Numerical evaluation of bounds as a function of sample size

Extension of asymptotic results to finite-length scenarios

Abstract

A central problem in Binary Hypothesis Testing (BHT) is to determine the optimal tradeoff between the Type I error (referred to as false alarm) and Type II (referred to as miss) error. In this context, the exponential rate of convergence of the optimal miss error probability -- as the sample size tends to infinity -- given some (positive) restrictions on the false alarm probabilities is a fundamental question to address in theory. Considering the more realistic context of a BHT with a finite number of observations, this paper presents a new non-asymptotic result for the scenario with monotonic (sub-exponential decreasing) restriction on the Type I error probability, which extends the result presented by Strassen in 2009. Building on the use of concentration inequalities, we offer new upper and lower bounds to the optimal Type II error probability for the case of finite observations. Finally, the derived bounds are evaluated and interpreted numerically (as a function of the number samples) for some vanishing Type I error restrictions.