Asymptotic expansion of the risk of maximum likelihood estimator with respect to $\alpha$-divergence as a measure of the difficulty of specifying a parametric model -- with detailed proof

TL;DR

This paper derives an asymptotic expansion for the risk of maximum likelihood estimators using $eta$-divergence, linking it to the geometric properties of the model's parameter space, to assess model specification difficulty.

Contribution

It provides a detailed asymptotic expansion of the MLE risk with respect to $eta$-divergence, incorporating geometric insights of the parameter manifold.

Findings

Asymptotic risk expansion up to order $n^{-2}$ for MLE.

Risk expressed via geometric properties of the parameter space.

Framework to measure model specification difficulty.

Abstract

For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to $\alpha$-divergence, which includes the special cases of Kullback--Leibler divergence, the Hellinger distance and $\chi^2$ divergence. The asymptotic expansion of the risk is given with respect to sample sizes of up to order $n^{-2}$. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model. We attempt to measure the difficulty of specifying a model through this expansion.