Maximum Likelihood under constraints: Degeneracies and Random Critical Points

TL;DR

This paper explores the behavior of maximum likelihood estimations under constraints when the estimating equations are mis-specified, revealing degeneracies, anomalous statistics, and establishing Bayesian posterior consistency.

Contribution

It introduces a detailed analysis of degeneracies and critical points in empirical likelihood under mis-specification, extending understanding beyond traditional null hypothesis scenarios.

Findings

Degeneracies in optimal distributions under mis-specification.

Log-likelihood Wilks statistic does not follow chi-squared distribution.

Bayesian procedures based on empirical likelihood are posterior consistent.

Abstract

We investigate the problem of semi-parametric maximum likelihood under constraints on summary statistics. Such a procedure results in a discrete probability distribution that maximises the likelihood among all such distributions under the specified constraints (called estimating equations), and is an approximation to the underlying population distribution. The study of such empirical likelihood originates from the seminal work of Owen. We investigate this procedure in the setting of mis-specified (or biased) estimating equations, i.e. when the null hypothesis is not true. We establish that the behaviour of the optimal distribution under such mis-specification differ markedly from their properties under the null, i.e. when the estimating equations are unbiased and correctly specified. This is manifested by certain degeneracies in the optimal distribution which define the likelihood. Such degeneracies are not observed under the null. Furthermore, we establish an anomalous behaviour of the log-likelihood based Wilks statistic, which, unlike under the null, does not exhibit a chi-squared limit. In the Bayesian setting, we rigorously establish the posterior consistency of procedures based on these ideas, where instead of a parametric likelihood, an empirical likelihood is used to define the posterior distribution. In particular, we show that this posterior, as a random probability measure, rapidly converges to the delta measure at the true parameter value. A novel feature of our approach is the investigation of critical points of random functions in the context of such empirical likelihood. In particular, we obtain the location and the mass of the degenerate optimal weights as the leading and sub-leading terms in a canonical expansion of a particular critical point of a random function that is naturally associated with the model.