On the last time and the number of times an estimator is more than epsilon from its target value

TL;DR

This paper studies the asymptotic behavior of the last time and the number of times an estimator deviates from its target by at least epsilon, providing limit distributions and applications to confidence sets and tests.

Contribution

It derives limit distributions for the last deviation time and count for estimators under broad conditions, including parametric and nonparametric cases, and introduces new optimality and construction methods.

Findings

Limit distributions for ${ m ext{ extsterling}}^2 N_ ext{ extsterling}$ and ${ m ext{ extsterling}}^2 Q_ ext{ extsterling}$ as epsilon approaches zero.

Results extend to non-i.i.d. situations and nonparametric density estimation.

New properties for maximum likelihood estimators and methods for sequential confidence sets and tests.

Abstract

Suppose $\widehat\theta_n$ is a strongly consistent estimator for $\theta_0$ in some i.i.d. situation. Let $N_\varepsilon$ and $Q_\varepsilon$ be respectively the last $n$ and the total number of $n$ for which $\widehat\theta_n$ is at least $\varepsilon$ away from $\theta_0$. The limit distributions for ${\varepsilon}^2 N_\varepsilon$ and ${\varepsilon}^2 Q_\varepsilon$ as $\varepsilon$ goes to zero are obtained under natural and weak conditions. The theory covers both parametric and nonparametric cases, multi-dimensional parameters, and general distance functions. Our results are of probabilistic interest, and, on the statistical side, suggest ways in which competing estimators can be compared. In particular several new optimality properties for the maximum likelihood estimator sequence in parametric families are established. Another use of our results is ways of constructing sequential fixed-volume or shrinking-volume confidence sets, as well as sequential tests with power 1. The paper also includes limit distribution results for the last $n$ and the number of $n$ for which the supremum distance $\|F_n-F\|\ge\varepsilon$, where $F_n$ is the empirical distribution function. Yet other results are reached for $\varepsilon^{5/2} N_\varepsilon$ and $\varepsilon^{5/2} Q_\varepsilon$ in the context of nonparametric density estimation, referring to the last time and the number of times where $|f_n(x) f(x)|\ge\varepsilon$. Finally it is shown that our results extend to several non-i.i.d. situations.