Fast sparse optimization via adaptive shrinkage

TL;DR

This paper introduces an adaptive proximal method based on logarithmic regularization that significantly accelerates the convergence of sparse optimization algorithms like Lasso, maintaining simplicity while improving speed.

Contribution

It develops and analyzes a novel adaptive shrinkage algorithm that enhances convergence speed in linear sparse optimization problems.

Findings

The proposed method converges faster than traditional algorithms.

Numerical experiments validate improved convergence speed.

Performance is competitive with state-of-the-art algorithms.

Abstract

The need for fast sparse optimization is emerging, e.g., to deal with large-dimensional data-driven problems and to track time-varying systems. In the framework of linear sparse optimization, the iterative shrinkage-thresholding algorithm is a valuable method to solve Lasso, which is particularly appreciated for its ease of implementation. Nevertheless, it converges slowly. In this paper, we develop a proximal method, based on logarithmic regularization, which turns out to be an iterative shrinkage-thresholding algorithm with adaptive shrinkage hyperparameter. This adaptivity substantially enhances the trajectory of the algorithm, in a way that yields faster convergence, while keeping the simplicity of the original method. Our contribution is twofold: on the one hand, we derive and analyze the proposed algorithm; on the other hand, we validate its fast convergence via numerical experiments and we discuss the performance with respect to state-of-the-art algorithms.