On some algorithms for estimation in Gaussian graphical models

TL;DR

This paper compares two algorithms for estimating parameters in Gaussian graphical models, highlighting their efficiency, convergence properties, and applicability to large, sparse graphs, with implications for maximum likelihood estimation.

Contribution

It introduces a new neighborhood coordinate descent algorithm for Gaussian graphical models and provides a simplified proof for the existence of the MLE in certain cases.

Findings

Iterative proportional scaling is feasible for large sparse graphs with simple convergence.

Neighborhood coordinate descent is extremely fast and less dependent on sparsity.

A method for finding positive definite starting values improves convergence.

Abstract

In Gaussian graphical models, the likelihood equations must typically be solved iteratively. We investigate two algorithms: A version of iterative proportional scaling which avoids inversion of large matrices, and an algorithm based on convex duality and operating on the covariance matrix by neighbourhood coordinate descent, corresponding to the graphical lasso with zero penalty. For large, sparse graphs, the iterative proportional scaling algorithm appears feasible and has simple convergence properties. The algorithm based on neighbourhood coordinate descent is extremely fast and less dependent on sparsity, but needs a positive definite starting value to converge. We give an algorithm for finding such a starting value for graphs with low colouring number. As a consequence, we also obtain a simplified proof for existence of the maximum likelihood estimator in such cases.