Exponential bounds for minimum contrast estimators

TL;DR

This paper develops exponential bounds for minimum contrast estimators that are applicable even with unbounded parameter sets, model misspecification, and small samples, providing concentration inequalities and risk bounds.

Contribution

It introduces a novel approach to exponential risk bounds for minimum contrast estimators that do not depend on entropy or covering numbers, applicable in unbounded and misspecified models.

Findings

Provides concentration inequalities for estimators.

Derives bounds useful for confidence set construction.

Illustrates results with i.i.d. samples and examples like change point estimation.

Abstract

The paper focuses on general properties of parametric minimum contrast estimators. The quality of estimation is measured in terms of the rate function related to the contrast, thus allowing to derive exponential risk bounds invariant with respect to the detailed probabilistic structure of the model. This approach works well for small or moderate samples and covers the case of a misspecified parametric model. Another important feature of the presented bounds is that they may be used in the case when the parametric set is unbounded and non-compact. These bounds do not rely on the entropy or covering numbers and can be easily computed. The most important statistical fact resulting from the exponential bonds is a concentration inequality which claims that minimum contrast estimators concentrate with a large probability on the level set of the rate function. In typical situations, every such set is a root-n neighborhood of the parameter of interest. We also show that the obtained bounds can help for bounding the estimation risk, constructing confidence sets for the underlying parameters. Our general results are illustrated for the case of an i.i.d. sample. We also consider several popular examples including least absolute deviation estimation and the problem of estimating the location of a change point. What we obtain in these examples slightly differs from the usual asymptotic results presented in statistical literature. This difference is due to the unboundness of the parameter set and a possible model misspecification.