BLOC: A Global Optimization Framework for Sparse Covariance Estimation with Non-Convex Penalties

TL;DR

BLOC is a versatile, gradient-free optimization framework for sparse covariance estimation that handles non-convex penalties, guarantees convergence, and outperforms existing methods in accuracy and sparsity recovery.

Contribution

It introduces a novel manifold reparameterization and a parallel, pattern search-based optimization approach for robust, penalty-agnostic sparse covariance estimation.

Findings

BLOC achieves lower estimation error than existing methods.

It ensures valid correlation matrices at every iteration.

Empirical results show improved sparsity recovery and scalability.

Abstract

We introduce BLOC (Black-box Optimization over Correlation matrices), a general framework for sparse covariance estimation with non-convex penalties. BLOC operates on the manifold of correlation matrices and reparameterizes it via an angular Cholesky mapping, transforming the positive-definite, unit-diagonal constraint into an unconstrained search over a Euclidean hyperrectangle. This enables gradient-free global optimization of diverse objectives, including non-differentiable or black-box losses, using a pattern search routine with adaptive coordinate polling, run-wise restarts to escape local minima, and leveraging up to $d(d-1)$ parallel threads when optimizing a $d$-dimensional correlation matrix. The method is penalty-agnostic and ensures that every iterate is a valid correlation matrix, from which covariance estimates are obtained. We establish convergence guarantees, including stationarity, probabilistic escape from poor local minima, and sublinear rates under smooth convex losses. From a statistical perspective, we prove consistency, convergence rates, and sparsistency for penalized correlation estimators under general conditions, extending sparse covariance theory beyond the Gaussian setting. Empirically, BLOC with nonconvex penalties such as SCAD and MCP outperforms leading estimators in both low- and high-dimensional regimes, achieving lower estimation error and improved sparsity recovery. A parallel implementation enhances scalability, and a proteomic network application demonstrates robust, positive-definite sparse covariance estimation.