An information-theoretic lower bound in time-uniform estimation

TL;DR

This paper establishes a fundamental information-theoretic lower bound for parameter estimation with time-uniform guarantees, revealing the inherent difficulty of such problems and providing sharp bounds for common models.

Contribution

It introduces a new reduction to sequential testing to derive stronger lower bounds for time-uniform estimation problems, applicable to various models.

Findings

Lower bound of ((( log log n)) for location, logistic regression, and exponential family models.

Bounds are sharp to within constant factors in typical settings.

Provides a theoretical foundation for understanding the hardness of time-uniform estimation.

Abstract

We present an information-theoretic lower bound for the problem of parameter estimation with time-uniform coverage guarantees. Via a new a reduction to sequential testing, we obtain stronger lower bounds that capture the hardness of the time-uniform setting. In the case of location model estimation, logistic regression, and exponential family models, our $\Omega(\sqrt{n^{-1}\log \log n})$ lower bound is sharp to within constant factors in typical settings.