A Hamilton-Jacobi-based Proximal Operator

TL;DR

This paper introduces HJ-Prox, a novel algorithm that approximates proximal operators for functions where explicit formulas are unknown, leveraging Hamilton-Jacobi equations and Monte Carlo methods, applicable even with noisy blackbox samples.

Contribution

The paper presents HJ-Prox, a new method for approximating proximal operators using Hamilton-Jacobi equations, extending applicability to blackbox and noisy functions.

Findings

HJ-Prox accurately approximates proximals and their gradients.

The method effectively denoises through adjustable smoothness.

Numerical examples demonstrate HJ-Prox's effectiveness.

Abstract

First-order optimization algorithms are widely used today. Two standard building blocks in these algorithms are proximal operators (proximals) and gradients. Although gradients can be computed for a wide array of functions, explicit proximal formulas are only known for limited classes of functions. We provide an algorithm, HJ-Prox, for accurately approximating such proximals. This is derived from a collection of relations between proximals, Moreau envelopes, Hamilton-Jacobi (HJ) equations, heat equations, and Monte Carlo sampling. In particular, HJ-Prox smoothly approximates the Moreau envelope and its gradient. The smoothness can be adjusted to act as a denoiser. Our approach applies even when functions are only accessible by (possibly noisy) blackbox samples. We show HJ-Prox is effective numerically via several examples.