Maximum Likelihood Estimation for Linear Gaussian Covariance Models

TL;DR

This paper analyzes the behavior of maximum likelihood estimation in linear Gaussian covariance models, showing that it often behaves like a convex problem for sample sizes roughly 14 times the dimension, despite the non-convex nature.

Contribution

The paper provides new theoretical conditions ensuring convergence to the global maximum in non-convex MLE problems for Gaussian covariance models, supported by numerical evidence.

Findings

MLE behaves as convex for n ≈ 14p

Conditions guarantee convergence to global maximum

Results hold even for small p, like p=2

Abstract

We study parameter estimation in linear Gaussian covariance models, which are $p$-dimensional Gaussian models with linear constraints on the covariance matrix. Maximum likelihood estimation for this class of models leads to a non-convex optimization problem which typically has many local maxima. Using recent results on the asymptotic distribution of extreme eigenvalues of the Wishart distribution, we provide sufficient conditions for any hill-climbing method to converge to the global maximum. Although we are primarily interested in the case in which $n>\!\!>p$, the proofs of our results utilize large-sample asymptotic theory under the scheme $n/p \to \gamma > 1$. Remarkably, our numerical simulations indicate that our results remain valid for $p$ as small as $2$. An important consequence of this analysis is that for sample sizes $n \simeq 14 p$, maximum likelihood estimation for linear Gaussian covariance models behaves as if it were a convex optimization problem.