Operator Splitting with Hamilton-Jacobi-based Proximals

TL;DR

This paper introduces a Hamilton-Jacobi-based proximal operator (HJ-Prox) framework that enables operator splitting methods to be used with functions lacking closed-form proximal operators, maintaining convergence guarantees.

Contribution

The authors develop a unified HJ-Prox framework for operator splitting, allowing the use of derivative-free, numerical proximal approximations in various algorithms.

Findings

HJ-Prox is competitive in statistical learning tasks.

Replacing exact proximal steps with HJ-Prox preserves convergence.

The framework broadens the applicability of operator splitting methods.

Abstract

Operator splitting algorithms are a cornerstone of modern first-order optimization, decomposing complex problems into simpler subproblems solved via proximal operators. However, most functions lack closed-form proximal operators, which has long restricted these methods to a narrow set of problems. Hamilton-Jacobi-based proximal operator (HJ-Prox) is a recent derivative-free Monte Carlo technique based on Hamilton-Jacobi PDE theory, that approximates proximal operators numerically. In this work, we introduce a unified framework for operator splitting via HJ-Prox, which allows for deployment of operator splitting even when functions are not proximable. We prove that replacing exact proximal steps with HJ-Prox in algorithms such as proximal point, proximal gradient descent, Douglas-Rachford splitting, Davis-Yin splitting, and primal-dual hybrid gradient preserves convergence guarantees under mild assumptions. Numerical experiments demonstrate HJ-Prox is competitive and effective on a wide variety of statistical learning tasks.