Sparsity by Worst-Case Penalties

TL;DR

This paper introduces a new interpretation of sparse penalties like elastic-net and group-lasso, leading to a unified and highly efficient optimization strategy that improves accuracy and interpretability in sparse modeling.

Contribution

It provides a novel interpretation of sparse penalties and develops a unified, computationally efficient optimization method that enhances accuracy and support recovery.

Findings

The new approach is computationally efficient for small to medium problems.

The software achieves machine-precision accuracy rapidly.

Accurate support recovery improves interpretability of sparsity penalties.

Abstract

This paper proposes a new interpretation of sparse penalties such as the elastic-net and the group-lasso. Beyond providing a new viewpoint on these penalization schemes, our approach results in a unified optimization strategy. Our experiments demonstrate that this strategy, implemented on the elastic-net, is computationally extremely efficient for small to medium size problems. Our accompanying software solves problems very accurately, at machine precision, in the time required to get a rough estimate with competing state-of-the-art algorithms. We illustrate on real and artificial datasets that this accuracy is required to for the correctness of the support of the solution, which is an important element for the interpretability of sparsity-inducing penalties.