A Proximal Distance Algorithm for Likelihood-Based Sparse Covariance Estimation

TL;DR

This paper introduces a likelihood-based proximal distance algorithm for estimating sparse covariance matrices that avoids shrinkage bias, handles high-dimensional data, and outperforms existing methods in simulations and real datasets.

Contribution

It proposes a novel optimization approach using proximal distance and majorization-minimization for sparse covariance estimation, improving accuracy and convergence.

Findings

Outperforms competing methods in simulated experiments

Handles high-dimensional settings effectively

Provides more accurate dependency networks in real data

Abstract

This paper addresses the task of estimating a covariance matrix under a patternless sparsity assumption. In contrast to existing approaches based on thresholding or shrinkage penalties, we propose a likelihood-based method that regularizes the distance from the covariance estimate to a symmetric sparsity set. This formulation avoids unwanted shrinkage induced by more common norm penalties and enables optimization of the resulting non-convex objective by solving a sequence of smooth, unconstrained subproblems. These subproblems are generated and solved via the proximal distance version of the majorization-minimization principle. The resulting algorithm executes rapidly, gracefully handles settings where the number of parameters exceeds the number of cases, yields a positive definite solution, and enjoys desirable convergence properties. Empirically, we demonstrate that our approach outperforms competing methods by several metrics across a suite of simulated experiments. Its merits are illustrated on an international migration dataset and a classic case study on flow cytometry. Our findings suggest that the marginal and conditional dependency networks for the cell signalling data are more similar than previously concluded.