A complete characterization of testable hypotheses

TL;DR

This paper fully characterizes when nontrivial hypothesis tests exist between two sets of probability measures, extending classical results to more general, nonparametric scenarios by considering closures in finitely additive measures.

Contribution

It completes Le Cam's program by providing a necessary and sufficient condition for testability without the common dominating measure assumption.

Findings

Characterizes testability using closures of convex hulls in finitely additive measures.

Extends classical total-variation separation results to nonparametric settings.

Provides examples and discusses measure-theoretic subtleties.

Abstract

We revisit a fundamental question in hypothesis testing: given two sets of probability measures $\mathcal{P}$ and $\mathcal{Q}$, when does a nontrivial (i.e. strictly unbiased) test for $\mathcal{P}$ against $\mathcal{Q}$ exist? Le Cam showed that, when $\mathcal{P}$ and $\mathcal{Q}$ have a common dominating measure, a test that has power exceeding its level by more than $\varepsilon$ exists if and only if the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ are separated in total-variation distance by more than $\varepsilon$. The requirement of a dominating measure is frequently violated in nonparametric statistics. In a passing remark, Le Cam described an approach to address more general scenarios, but he stopped short of stating a formal theorem. This work completes Le Cam's program, by presenting a matching necessary and sufficient condition for testability: for the aforementioned theorem to hold without assumptions, one must take the closures of the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ in the space of bounded finitely additive measures. We provide simple elucidating examples, and elaborate on various subtle measure theoretic and topological points regarding compactness and achievability.