Sparse Inverse Covariance Selection via Alternating Linearization Methods

TL;DR

This paper introduces a first-order alternating linearization method for efficiently estimating sparse inverse covariance matrices in Gaussian graphical models, achieving optimal convergence and outperforming existing algorithms in experiments.

Contribution

The paper presents a novel alternating linearization algorithm tailored for sparse inverse covariance estimation, with closed-form subproblems and optimal iteration complexity.

Findings

Algorithm converges in O(1/ε) iterations.

Outperforms other methods in synthetic and real gene network data.

Provides practical and efficient solution for structure learning.

Abstract

Gaussian graphical models are of great interest in statistical learning. Because the conditional independencies between different nodes correspond to zero entries in the inverse covariance matrix of the Gaussian distribution, one can learn the structure of the graph by estimating a sparse inverse covariance matrix from sample data, by solving a convex maximum likelihood problem with an $\ell_1$-regularization term. In this paper, we propose a first-order method based on an alternating linearization technique that exploits the problem's special structure; in particular, the subproblems solved in each iteration have closed-form solutions. Moreover, our algorithm obtains an $\epsilon$-optimal solution in $O(1/\epsilon)$ iterations. Numerical experiments on both synthetic and real data from gene association networks show that a practical version of this algorithm outperforms other competitive algorithms.