Decreasing Weighted Sorted $\ell_1$ Regularization

TL;DR

This paper introduces a new family of regularizers called weighted sorted $ ext{l}_1$ norms (WSL1), generalizing existing norms and focusing on a decreasing variant (DWSL1) with proven norm properties and derived computational tools.

Contribution

The paper defines the decreasing WSL1 norm, proves it is a valid norm, and derives its dual norm and proximity operator for regularization applications.

Findings

DWSL1 is a valid norm.

Derived dual norm and proximity operator for DWSL1.

Generalizes OSCAR, $ ext{l}_1$, and $ ext{l}_ ext{infinity}$ norms.

Abstract

We consider a new family of regularizers, termed {\it weighted sorted $\ell_1$ norms} (WSL1), which generalizes the recently introduced {\it octagonal shrinkage and clustering algorithm for regression} (OSCAR) and also contains the $\ell_1$ and $\ell_{\infty}$ norms as particular instances. We focus on a special case of the WSL1, the {\sl decreasing WSL1} (DWSL1), where the elements of the argument vector are sorted in non-increasing order and the weights are also non-increasing. In this paper, after showing that the DWSL1 is indeed a norm, we derive two key tools for its use as a regularizer: the dual norm and the Moreau proximity operator.