Sparse Matrix Inversion with Scaled Lasso

TL;DR

This paper introduces a scaled Lasso-based method for estimating sparse inverse covariance matrices, achieving faster convergence rates and improved theoretical guarantees without cross-validation, demonstrated through simulations.

Contribution

The paper presents a novel scaled Lasso algorithm for sparse matrix inversion with theoretical convergence guarantees and practical efficiency, surpassing existing methods.

Findings

Faster convergence rates under weaker conditions.

Data-driven penalty level determination without cross-validation.

Superior performance demonstrated in simulations.

Abstract

We propose a new method of learning a sparse nonnegative-definite target matrix. Our primary example of the target matrix is the inverse of a population covariance or correlation matrix. The algorithm first estimates each column of the target matrix by the scaled Lasso and then adjusts the matrix estimator to be symmetric. The penalty level of the scaled Lasso for each column is completely determined by data via convex minimization, without using cross-validation. We prove that this scaled Lasso method guarantees the fastest proven rate of convergence in the spectrum norm under conditions of weaker form than those in the existing analyses of other $\ell_1$ regularized algorithms, and has faster guaranteed rate of convergence when the ratio of the $\ell_1$ and spectrum norms of the target inverse matrix diverges to infinity. A simulation study demonstrates the computational feasibility and superb performance of the proposed method. Our analysis also provides new performance bounds for the Lasso and scaled Lasso to guarantee higher concentration of the error at a smaller threshold level than previous analyses, and to allow the use of the union bound in column-by-column applications of the scaled Lasso without an adjustment of the penalty level. In addition, the least squares estimation after the scaled Lasso selection is considered and proven to guarantee performance bounds similar to that of the scaled Lasso.