Implicit Differentiation for Hyperparameter Tuning the Weighted Graphical Lasso

TL;DR

This paper introduces a novel framework using implicit differentiation to efficiently tune hyperparameters in the Weighted Graphical Lasso, enabling more accurate graphical model estimation.

Contribution

It derives the Jacobian of the Graphical Lasso solution with respect to hyperparameters and proposes a bilevel optimization approach solved with a first-order method.

Findings

Effective hyperparameter tuning for Weighted Graphical Lasso

Improved accuracy in graphical model estimation

Efficient first-order optimization algorithm

Abstract

We provide a framework and algorithm for tuning the hyperparameters of the Graphical Lasso via a bilevel optimization problem solved with a first-order method. In particular, we derive the Jacobian of the Graphical Lasso solution with respect to its regularization hyperparameters.