Sparse Regularization via Convex Analysis

TL;DR

This paper introduces a novel class of non-convex penalty functions for sparse solutions that preserve convexity of the optimization problem, avoiding L1 underestimation and enabling efficient minimization.

Contribution

It proposes a multivariate generalization of the MC penalty and Huber function, maintaining convexity while improving sparse solution accuracy.

Findings

The new penalty maintains convexity of the least squares problem.

It avoids the systematic underestimation of L1 regularization.

The cost function can be minimized efficiently using proximal algorithms.

Abstract

Sparse approximate solutions to linear equations are classically obtained via L1 norm regularized least squares, but this method often underestimates the true solution. As an alternative to the L1 norm, this paper proposes a class of non-convex penalty functions that maintain the convexity of the least squares cost function to be minimized, and avoids the systematic underestimation characteristic of L1 norm regularization. The proposed penalty function is a multivariate generalization of the minimax-concave (MC) penalty. It is defined in terms of a new multivariate generalization of the Huber function, which in turn is defined via infimal convolution. The proposed sparse-regularized least squares cost function can be minimized by proximal algorithms comprising simple computations.