Sparse Prediction with the $k$-Support Norm

TL;DR

The paper introduces the $k$-support norm, a new convex regularizer that offers a tighter relaxation for sparse prediction than elastic net, improving upon Lasso and elastic net methods.

Contribution

It derives the $k$-support norm as the tightest convex relaxation of sparsity combined with an $\\ell_2$ penalty, providing a better alternative for sparse prediction tasks.

Findings

$k$-support norm provides a tighter relaxation than elastic net.

Bound on the looseness of elastic net established.

Justifies the use of elastic net through new bounds.

Abstract

We derive a novel norm that corresponds to the tightest convex relaxation of sparsity combined with an $\ell_2$ penalty. We show that this new {\em $k$-support norm} provides a tighter relaxation than the elastic net and is thus a good replacement for the Lasso or the elastic net in sparse prediction problems. Through the study of the $k$-support norm, we also bound the looseness of the elastic net, thus shedding new light on it and providing justification for its use.