L1 Regularization Paths in Linear Models by Parametric Gaussian Message Passing

TL;DR

This paper introduces two novel algorithms for computing L1 regularization paths in linear models using parametric Gaussian message passing, applicable to various settings like Kalman smoothing and SVM.

Contribution

The paper presents dual algorithms leveraging Gaussian message passing for L1 regularization paths, broadening applicability and potentially improving computational efficiency.

Findings

Algorithms are broadly applicable to different L1 regularization problems.

Methods mainly require matrix multiplications, making them computationally efficient.

Complexity can be competitive with existing methods in certain cases.

Abstract

The paper considers the computation of L1 regularization paths in a state space setting, which includes L1 regularized Kalman smoothing, linear SVM, LASSO, and more. The paper proposes two new algorithms, which are duals of each other; the first algorithm applies to L1 regularization of independent variables while the second applies to L1 regularization of dependent variables. The heart of the proposed algorithms is parametric Gaussian message passing (i.e., Kalman-type forward-backward recursions) in the pertinent factor graphs. The proposed methods are broadly applicable, they (usually) require only matrix multiplications, and their complexity can be competitive with prior methods in some cases.