Sparse Gaussian Graphical Models with Discrete Optimization: Computational and Statistical Perspectives

TL;DR

This paper introduces GraphL0BnB, a novel estimator for sparse Gaussian graphical models using an $$-penalized pseudo-likelihood, solved via a custom nonlinear branch-and-bound framework, with strong statistical guarantees and improved computational efficiency.

Contribution

It proposes a new $$-penalized estimator formulated as a mixed integer program and develops a specialized branch-and-bound algorithm for efficient solution, advancing sparse graphical model learning.

Findings

The BnB framework outperforms commercial solvers in runtime and accuracy.

The estimator provides strong statistical guarantees for estimation and variable selection.

Numerical experiments show improved performance over existing methods.

Abstract

We consider the problem of learning a sparse graph underlying an undirected Gaussian graphical model, a key problem in statistical machine learning. Given $n$ samples from a multivariate Gaussian distribution with $p$ variables, the goal is to estimate the $p \times p$ inverse covariance matrix (aka precision matrix), assuming it is sparse (i.e., has a few nonzero entries). We propose GraphL0BnB, a new estimator based on an $\ell_0$-penalized version of the pseudo-likelihood function, while most earlier approaches are based on the $\ell_1$-relaxation. Our estimator can be formulated as a convex mixed integer program (MIP) which can be difficult to compute beyond $p\approx 100$ using off-the-shelf commercial solvers. To solve the MIP, we propose a custom nonlinear branch-and-bound (BnB) framework that solves node relaxations with tailored first-order methods. As a key component of our BnB framework, we propose large-scale solvers for obtaining good primal solutions that are of independent interest. We derive novel statistical guarantees (estimation and variable selection) for our estimator and discuss how our approach improves upon existing estimators. Our numerical experiments on real and synthetic datasets suggest that our BnB framework offers significant advantages over off-the-shelf commercial solvers, and our approach has favorable performance (both in terms of runtime and statistical performance) compared to the state-of-the-art approaches for learning sparse graphical models.