Approximation of the Proximal Operator of the $\ell_\infty$ Norm Using a Neural Network

TL;DR

This paper introduces a neural network-based method to approximate the proximal operator of the infinity norm efficiently, avoiding sorting operations, and demonstrates its accuracy and computational benefits over traditional methods.

Contribution

The authors develop an $O(m)$ neural network approximation for the proximal operator of the infinity norm that handles variable input sizes using feature selection.

Findings

The neural network outperforms vanilla neural networks in approximation accuracy.

The proposed method is computationally more efficient than exact algorithms.

Feature importance analysis shows effective selection of input data features.

Abstract

Computing the proximal operator of the $\ell_\infty$ norm, $\textbf{prox}_{\alpha ||\cdot||_\infty}(\mathbf{x})$, generally requires a sort of the input data, or at least a partial sort similar to quicksort. In order to avoid using a sort, we present an $O(m)$ approximation of $\textbf{prox}_{\alpha ||\cdot||_\infty}(\mathbf{x})$ using a neural network. A novel aspect of the network is that it is able to accept vectors of varying lengths due to a feature selection process that uses moments of the input data. We present results on the accuracy of the approximation, feature importance, and computational efficiency of the approach. We show that the network outperforms a "vanilla neural network" that does not use feature selection. We also present an algorithm with corresponding theory to calculate $\textbf{prox}_{\alpha ||\cdot||_\infty}(\mathbf{x})$ exactly, relate it to the Moreau decomposition, and compare its computational efficiency to that of the approximation.