Quasi-maximum likelihood estimation and penalized estimation under non-standard conditions

TL;DR

This paper develops a comprehensive estimation theory for non-regular models, including penalized and quasi-maximum likelihood estimations, accommodating boundary parameters and non-ergodic statistics with complex limit distributions.

Contribution

It introduces a general local approximation of the parameter space for non-regular models, enabling derivation of limit distributions and analysis of penalized estimators under boundary constraints.

Findings

Established limit distribution results for non-regular models.

Provided conditions for selection consistency of penalized estimators.

Demonstrated applications to GIG distribution, non-ergodic diffusion, and variance components in mixed models.

Abstract

The purpose of this article is to develop a general parametric estimation theory that allows the derivation of the limit distribution of estimators in non-regular models where the true parameter value may lie on the boundary of the parameter space or where even identifiability fails. For that, we propose a more general local approximation of the parameter space (at the true value) than previous studies. This estimation theory is comprehensive in that it can handle penalized estimation as well as quasi-maximum likelihood estimation under such non-regular models. Besides, our results can apply to the so-called non-ergodic statistics, where the Fisher information is random in the limit, including the regular experiment that is locally asymptotically mixed normal. In penalized estimation, depending on the boundary constraint, even the Bridge estimator with $q<1$ does not necessarily give selection consistency. Therefore, some sufficient condition for selection consistency is described, precisely evaluating the balance between the boundary constraint and the form of the penalty. Examples handled in the paper are: (i) ML estimation of the generalized inverse Gaussian distribution, (ii) quasi-ML estimation of the diffusion parameter in a non-ergodic It\^o process whose parameter space consists of positive semi-definite symmetric matrices, while the drift parameter is treated as nuisance and (iii) penalized ML estimation of variance components of random effects in linear mixed models.