Parameter Choices for Sparse Regularization with the $\ell_1$ Norm

TL;DR

This paper provides a theoretical analysis of how the regularization parameter influences sparsity in $\,\ell_1$-norm regularized problems, offering insights for optimal parameter selection to promote desired sparsity levels.

Contribution

It characterizes the sparsity of solutions based on the transform matrix and proposes strategies for choosing regularization parameters to control sparsity and address ill-posedness.

Findings

Characterization of sparsity depending on the transform matrix.

Regularization parameter choices can balance sparsity and fidelity.

Numerical experiments validate the theoretical insights.

Abstract

We consider a regularization problem whose objective function consists of a convex fidelity term and a regularization term determined by the $\ell_1$ norm composed with a linear transform. Empirical results show that the regularization with the $\ell_1$ norm can promote sparsity of a regularized solution. It is the goal of this paper to understand theoretically the effect of the regularization parameter on the sparsity of the regularized solutions. We establish a characterization of the sparsity under the transform matrix of the solution. When the fidelity term has a special structure and the transform matrix coincides with a identity matrix, the resulting characterization can be taken as a regularization parameter choice strategy with which the regularization problem has a solution having a sparsity of a certain level. We study choices of the regularization parameter so that the regularization term alleviates the ill-posedness and promote sparsity of the resulting regularized solution. Numerical experiments demonstrate that choices of the regularization parameters can balance the sparsity of the solutions of the regularization problem and its approximation to the minimizer of the fidelity function.