Sparse Regularization with the $\ell_0$ Norm

TL;DR

This paper analyzes how to select the regularization parameter in sparse regularization problems involving the $\,\ell_0$ norm to control the sparsity level of solutions, with applications in compressed sensing and image processing.

Contribution

It introduces a geometric framework to determine regularization parameters that achieve desired sparsity levels in solutions, especially for the identity transform case.

Findings

Provides conditions for regularization parameters to ensure prescribed sparsity levels.

Develops a geometric viewpoint to understand sparsity partitioning in the solution space.

Applies results to practical scenarios like machine learning and medical imaging.

Abstract

We consider a minimization problem whose objective function is the sum of a fidelity term, not necessarily convex, and a regularization term defined by a positive regularization parameter $\lambda$ multiple of the $\ell_0$ norm composed with a linear transform. This problem has wide applications in compressed sensing, sparse machine learning and image reconstruction. The goal of this paper is to understand what choices of the regularization parameter can dictate the level of sparsity under the transform for a global minimizer of the resulting regularized objective function. This is a critical issue but it has been left unaddressed. We address it from a geometric viewpoint with which the sparsity partition of the image space of the transform is introduced. Choices of the regularization parameter are specified to ensure that a global minimizer of the corresponding regularized objective function achieves a prescribed level of sparsity under the transform. Results are obtained for the spacial sparsity case in which the transform is the identity map, a case that covers several applications of practical importance, including machine learning, image/signal processing and medical image reconstruction.