Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator

TL;DR

This paper enhances classical asymptotic normality results for the MLE by establishing sub-Gaussian tail bounds, entropic convergence, and conditions for normality without smoothing, using novel auxiliary tools.

Contribution

It introduces stronger forms of asymptotic normality for the MLE, including sub-Gaussian bounds and entropic CLTs, under new assumptions and with refined techniques.

Findings

Sub-Gaussian tail bounds for normalized MLE

Entropic central limit theorem for smoothed estimator

Normality of MLE without smoothing under bounded Fisher information

Abstract

It is well known that, under standard regularity conditions, the maximum likelihood estimator (MLE) satisfies a central limit theorem and converges in distribution to a Gaussian random variable as the sample size grows. This paper strengthens this classical result by developing several stronger forms of asymptotic normality for the normalized MLE. With additional assumptions on the score, we first establish sub-Gaussian tail bounds and convergence of all moments for the normalized estimation error. We then prove an entropic central limit theorem for a smoothed version of the estimator, showing convergence in relative entropy to the limiting Gaussian law. When the Fisher information of the normalized estimate is bounded, or its density has bounded first derivative, we further show that the smoothing can be removed, yielding entropic normality of the MLE itself. The proofs develop auxiliary tools that may be of independent interest, including exponential consistency bounds, high-moment estimates, and entropy-control arguments for the estimator.