Penalized maximum likelihood estimation and effective dimension

TL;DR

This paper extends classical statistical results like Fisher and Wilks theorems to penalized maximum likelihood estimation with quadratic penalties, providing sharp, non-asymptotic expansions based on the effective dimension.

Contribution

It introduces non-asymptotic expansions for penalized MLE and likelihood, emphasizing the role of effective dimension, applicable even in infinite-dimensional settings.

Findings

Fisher expansion holds when p_G^2/n is small

Wilks remainder is of order p_G^3/n

Results do not rely on asymptotic arguments

Abstract

This paper extends some prominent statistical results including \emph{Fisher Theorem and Wilks phenomenon} to the penalized maximum likelihood estimation with a quadratic penalization. It appears that sharp expansions for the penalized MLE \(\tilde{\thetav}_{G} \) and for the penalized maximum likelihood can be obtained without involving any asymptotic arguments, the results only rely on smoothness and regularity properties of the of the considered log-likelihood function. The error of estimation is specified in terms of the effective dimension \(p_G \) of the parameter set which can be much smaller than the true parameter dimension and even allows an infinite dimensional functional parameter. In the i.i.d. case, the Fisher expansion for the penalized MLE can be established under the constraint "\(p_G^{2}/n\) is small" while the remainder in the Wilks result is of order \(p_G^{3}/n \).