On Gaussian Approximation for M-Estimator

TL;DR

This paper introduces a non-asymptotic Gaussian approximation framework for M-estimators, enabling better understanding and bootstrap methods for their distribution, including non-regular cases.

Contribution

It extends Gaussian approximation theory to the maximizers of empirical criteria, covering both regular and non-regular M-estimators.

Findings

Developed non-asymptotic Gaussian approximation results.

Proposed a Gaussian multiplier bootstrap method.

Applicable to regular and non-regular M-estimators.

Abstract

This study develops a non-asymptotic Gaussian approximation theory for distributions of M-estimators, which are defined as maximizers of empirical criterion functions. In existing mathematical statistics literature, numerous studies have focused on approximating the distributions of the M-estimators for statistical inference. In contrast to the existing approaches, which mainly focus on limiting behaviors, this study employs a non-asymptotic approach, establishes abstract Gaussian approximation results for maximizers of empirical criteria, and proposes a Gaussian multiplier bootstrap approximation method. Our developments can be considered as extensions of the seminal works (Chernozhukov, Chetverikov and Kato (2013, 2014, 2015)) on the approximation theory for distributions of suprema of empirical processes toward their maximizers. Through this work, we shed new lights on the statistical theory of M-estimators. Our theory covers not only regular estimators, such as the least absolute deviations, but also some non-regular cases where it is difficult to derive or to approximate numerically the limiting distributions such as non-Donsker classes and cube root estimators.