A block-coordinate descent framework for non-convex composite optimization. Application to sparse precision matrix estimation

TL;DR

This paper introduces a versatile block-coordinate descent framework for non-convex composite optimization, providing convergence guarantees and significantly reducing iterations in sparse precision matrix estimation.

Contribution

It presents a general BCD framework applicable to various solvers, with theoretical convergence and practical efficiency improvements for sparse precision matrix estimation.

Findings

Convergence guarantees for the proposed BCD framework.

Up to 100-fold reduction in iterations for estimation quality.

Applicability to multiple popular solvers in sparse precision matrix estimation.

Abstract

Block-coordinate descent (BCD) is the method of choice to solve numerous large scale optimization problems, however their theoretical study for non-convex optimization, has received less attention. In this paper, we present a new block-coordinate descent (BCD) framework to tackle non-convex composite optimization problems, ensuring decrease of the objective function and convergence to a solution. This framework is general enough to include variable metric proximal gradient updates, proximal Newton updates, and alternated minimization updates. This generality allows to encompass three versions of the most used solvers in the sparse precision matrix estimation problem, deemed Graphical Lasso: graphical ISTA, Primal GLasso, and QUIC. We demonstrate the value of this new framework on non-convex sparse precision matrix estimation problems, providing convergence guarantees and up to a $100$-fold reduction in the number of iterations required to reach state-of-the-art estimation quality.