Capped Lp approximations for the composite L0 regularization problem

TL;DR

This paper introduces capped Lp approximations for the composite L0 regularization problem, providing theoretical insights into their convergence and optimality, which may guide the development of new algorithms for sparse regularization.

Contribution

It establishes the existence of solutions and the asymptotic equivalence between the capped Lp approximation and the original composite L0 problem under broad conditions.

Findings

Capped Lp functions converge pointwise to the L0 norm as the approximation parameter increases.

The capped Lp approximation acts as a penalty method for the composite L0 problem.

Under certain conditions, the original and approximated problems share the same optimal solutions.

Abstract

The composite L0 function serves as a sparse regularizer in many applications. The algorithmic difficulty caused by the composite L0 regularization (the L0 norm composed with a linear mapping) is usually bypassed through approximating the L0 norm. We consider in this paper capped Lp approximations with $p>0$ for the composite L0 regularization problem. For each $p>0$, the capped Lp function converges to the L0 norm pointwisely as the approximation parameter tends to infinity. We point out that the capped Lp approximation problem is essentially a penalty method with an Lp penalty function for the composite L0 problem from the viewpoint of numerical optimization. Our theoretical results stated below may shed a new light on the penalty methods for solving the composite L0 problem and help the design of innovative numerical algorithms. We first establish the existence of optimal solutions to the composite L0 regularization problem and its capped Lp approximation problem under conditions that the data fitting function is asymptotically level stable and bounded below. Asymptotically level stable functions cover a rich class of data fitting functions encountered in practice. We then prove that the capped Lp problem asymptotically approximates the composite L0 problem if the data fitting function is a level bounded function composed with a linear mapping. We further show that if the data fitting function is the indicator function on an asymptotically linear set or the L0 norm composed with an affine mapping, then the composite L0 problem and its capped Lp approximation problem share the same optimal solution set provided that the approximation parameter is large enough.