From Graphical Lasso to Atomic Norms: High-Dimensional Pattern Recovery

TL;DR

This paper extends high-dimensional pattern recovery analysis from graphical lasso to a broad class of atomic norms, providing theoretical guarantees and improved conditions for accurate precision matrix estimation.

Contribution

It introduces a unified framework for pattern recovery using atomic norms with polytope unit balls, refining irrepresentability conditions and enhancing theoretical guarantees over prior methods.

Findings

Established theoretical guarantees for atomic norm-based pattern recovery.

Derived weaker deviation and irrepresentability conditions for improved performance.

Numerical examples confirm the tightness of the theoretical bounds.

Abstract

Estimating high-dimensional precision matrices is a fundamental problem in modern statistics, with the graphical lasso and its $\ell_1$-penalty being a standard approach for recovering sparsity patterns. However, many statistical models, e.g. colored graphical models, exhibit richer structures like symmetry or equality constraints, which the $\ell_1$-norm cannot adequately capture. This paper addresses the gap by extending the high-dimensional analysis of pattern recovery to a general class of atomic norm penalties, particularly those whose unit balls are polytopes, where patterns correspond to the polytope's facial structure. We establish theoretical guarantees for recovering the true pattern induced by these general atomic norms in precision matrix estimation. Our framework builds upon and refines the primal-dual witness methodology of Ravikumar et al. (2011). Our analysis provides conditions on the deviation between sample and true covariance matrices for successful pattern recovery, given a novel, generalized irrepresentability condition applicable to any atomic norm. When specialized to the $\ell_1$-penalty, our results offer improved conditions -- including weaker deviation requirements and a less restrictive irrepresentability condition -- leading to tighter bounds and better asymptotic performance than prior work. The proposed general irrepresentability condition, based on a new thresholding concept, provides a unified perspective on model selection consistency. Numerical examples demonstrate the tightness of the derived theoretical bounds.