Fixed-horizon Active Hypothesis Testing

TL;DR

This paper formulates fixed-horizon active hypothesis testing problems, deriving bounds and proposing deterministic, adaptive strategies that outperform classical methods, especially in finite-sample scenarios.

Contribution

It introduces novel deterministic, adaptive experiment selection strategies for fixed-horizon active hypothesis testing, improving performance over classical randomized approaches.

Findings

Derived asymptotically tight bounds on misclassification probabilities.

Proposed asymptotically optimal deterministic, adaptive strategies.

Numerical experiments show superior non-asymptotic performance.

Abstract

Two active hypothesis testing problems are formulated. In these problems, the agent can perform a fixed number of experiments and then decide on one of the hypotheses. The agent is also allowed to declare its experiments inconclusive if needed. The first problem is an asymmetric formulation in which the the objective is to minimize the probability of incorrectly declaring a particular hypothesis to be true while ensuring that the probability of correctly declaring that hypothesis is moderately high. This formulation can be seen as a generalization of the formulation in the classical Chernoff-Stein lemma to an active setting. The second problem is a symmetric formulation in which the objective is to minimize the probability of making an incorrect inference (misclassification probability) while ensuring that the true hypothesis is declared conclusively with moderately high probability. For these problems, lower and upper bounds on the optimal misclassification probabilities are derived and these bounds are shown to be asymptotically tight. Classical approaches for experiment selection suggest use of randomized and, in some cases, open-loop strategies. As opposed to these classical approaches, fully deterministic and adaptive experiment selection strategies are provided. It is shown that these strategies are asymptotically optimal and further, using numerical experiments, it is demonstrated that these novel experiment selection strategies (coupled with appropriate inference strategies) have a significantly better performance in the non-asymptotic regime.