Reverse Information Projections and Optimal E-statistics

TL;DR

This paper extends the concept of reverse information projection (RIPr) to cases with infinite divergence, providing a natural way to construct optimal e-statistics for hypothesis testing even when traditional methods are undefined.

Contribution

It introduces an extended RIPr that applies when divergence is infinite, offering a new approach to optimal e-statistics beyond previous limitations.

Findings

Extended RIPr exists under certain conditions with infinite divergence.

The extended RIPr aligns with the classical RIPr when divergence is finite.

Conditions are provided for when the extended RIPr is a strict sub-probability measure.

Abstract

Information projections have found important applications in probability theory, statistics, and related areas. In the field of hypothesis testing in particular, the reverse information projection (RIPr) has recently been shown to lead to growth-rate optimal (GRO) e-statistics for testing simple alternatives against composite null hypotheses. However, the RIPr as well as the GRO criterion are undefined whenever the infimum information divergence between the null and alternative is infinite. We show that in such scenarios, under some assumptions, there still exists a measure in the null that is closest to the alternative in a specific sense. Whenever the information divergence is finite, this measure coincides with the usual RIPr. It therefore gives a natural extension of the RIPr to certain cases where the latter was previously not defined. This extended notion of the RIPr is shown to lead to optimal e-statistics in a sense that is a novel, but natural, extension of the GRO criterion. We also give conditions under which the (extension of the) RIPr is a strict sub-probability measure, as well as conditions under which an approximation of the RIPr leads to approximate e-statistics. For this case we provide tight relations between the corresponding approximation rates.