The fastest $\ell_{1,\infty}$ prox in the west

TL;DR

This paper introduces a fast, closed-form proximal operator for the mixed , norm, enabling efficient optimization in large-scale problems by applying column-wise soft-thresholding with adaptive thresholds.

Contribution

It derives a closed-form solution for the , norm proximal operator and proposes an iterative algorithm with efficient implementations for large-scale applications.

Findings

Proposed methods are significantly faster than existing algorithms.

The algorithms effectively handle large-scale synthetic and real data.

Adaptive thresholds improve the accuracy and efficiency of the proximal computation.

Abstract

Proximal operators are of particular interest in optimization problems dealing with non-smooth objectives because in many practical cases they lead to optimization algorithms whose updates can be computed in closed form or very efficiently. A well-known example is the proximal operator of the vector $\ell_1$ norm, which is given by the soft-thresholding operator. In this paper we study the proximal operator of the mixed $\ell_{1,\infty}$ matrix norm and show that it can be computed in closed form by applying the well-known soft-thresholding operator to each column of the matrix. However, unlike the vector $\ell_1$ norm case where the threshold is constant, in the mixed $\ell_{1,\infty}$ norm case each column of the matrix might require a different threshold and all thresholds depend on the given matrix. We propose a general iterative algorithm for computing these thresholds, as well as two efficient implementations that further exploit easy to compute lower bounds for the mixed norm of the optimal solution. Experiments on large-scale synthetic and real data indicate that the proposed methods can be orders of magnitude faster than state-of-the-art methods.