Asymptotic Expansion of Risk for a Regression Model with respect to $\alpha$-Divergence with an Application to the Sample Size Problem -- Complete Version

TL;DR

This paper derives the asymptotic expansion of the risk for maximum likelihood estimators in regression models with respect to α-divergence, analyzing how distributional factors influence convergence speed and sample size requirements.

Contribution

It provides the first detailed asymptotic expansion of the estimator's risk up to order n^{-2} for α-divergence, including concrete distributional cases and a method to estimate sample size needs.

Findings

Risk convergence speed depends on error distribution and joint moments of explanatory variables.

Explicit asymptotic risk expansion up to order n^{-2} for various error distributions.

A standard for sample size estimation based on risk comparison with binomial distribution.

Abstract

For a regression model, we consider the risk of the maximum likelihood estimator with respect to $\alpha$-divergence, which includes the special cases of Kullback-Leibler divergence, Hellinger distance and $\chi^2$ divergence. The asymptotic expansion of the risk with respect to the sample size $n$ is given up to the order $n^{-2}$. We are interested in how the risk convergence speed (to zero) is affected by the error term distributions of the regression model and the magnitude of the joint moments of the standardized explanatory variables. Besides the general result (which is given by Mathematica program), we consider three concrete error term distributions; a normal distribution, a t-distribution and a skew-normal distribution. We use the (approximated) risk of m.l.e. as a measure of the difficulty of estimation for the regression model. Especially comparing the value of the (approximated) risk with that of a binomial distribution, we can give a certain standard for the sample size required to estimate the regression model.