WEEP: A Differentiable Nonconvex Sparse Regularizer via Weakly-Convex Envelope

TL;DR

This paper introduces WEEP, a novel differentiable sparse regularizer based on weakly-convex envelopes, enabling effective sparse modeling with gradient-based methods and demonstrating superior performance in compressive sensing and image denoising.

Contribution

WEEP is a new differentiable regularizer derived from weakly-convex envelopes, offering unbiased sparsity, a closed-form proximal operator, and compatibility with gradient-based optimization.

Findings

WEEP outperforms existing regularizers in compressive sensing tasks.

WEEP achieves superior results in image denoising.

WEEP maintains full differentiability and L-smoothness.

Abstract

Sparse regularization is fundamental in signal processing and feature extraction but often relies on non-differentiable penalties, conflicting with gradient-based optimizers. We propose WEEP (Weakly-convex Envelope of Piecewise Penalty), a novel differentiable regularizer derived from the weakly-convex envelope framework. WEEP provides tunable, unbiased sparsity and a simple closed-form proximal operator, while maintaining full differentiability and L-smoothness, ensuring compatibility with both gradient-based and proximal algorithms. This resolves the tradeoff between statistical performance and computational tractability. We demonstrate superior performance compared to established convex and non-convex sparse regularizers on challenging compressive sensing and image denoising tasks.