On the errors committed by sequences of estimator functionals

TL;DR

This paper analyzes the asymptotic behavior of estimator sequences that converge almost surely, providing new insights into their errors, tail distributions, and confidence regions, especially for functional estimators in complex statistical settings.

Contribution

It extends existing results to functional estimators, deriving limit distributions for tail events and constructing sequential confidence regions under weak smoothness conditions.

Findings

Derived asymptotic limits for the last time estimators are beyond a threshold.

Constructed analytic approximations for sequential fixed-width confidence regions.

Applied results to a new confidence set for the cumulative hazard function and a stochastic programming problem.

Abstract

Consider a sequence of estimators $\hat \theta_n$ which converges almost surely to $\theta_0$ as the sample size $n$ tends to infinity. Under weak smoothness conditions, we identify the asymptotic limit of the last time $\hat \theta_n$ is further than $\eps$ away from $\theta_0$ when $\eps \rightarrow 0^+$. These limits lead to the construction of sequentially fixed width confidence regions for which we find analytic approximations. The smoothness conditions we impose is that $\hat \theta_n$ is to be close to a Hadamard-differentiable functional of the empirical distribution, an assumption valid for a large class of widely used statistical estimators. Similar results were derived in Hjort and Fenstad (1992, Annals of Statistics) for the case of Euclidean parameter spaces; part of the present contribution is to lift these results to situations involving parameter functionals. The apparatus we develop is also used to derive appropriate limit distributions of other quantities related to the far tail of an almost surely convergent sequence of estimators, like the number of times the estimator is more than $\eps$ away from its target. We illustrate our results by giving a new sequential simultaneous confidence set for the cumulative hazard function based on the Nelson--Aalen estimator and investigate a problem in stochastic programming related to computational complexity.