Mirror-Free Proximal Methods

TL;DR

This paper introduces a mirror-free variant of the mirror prox algorithm that simplifies proximal updates, extends convergence analysis under relative smoothness, and applies to complex min-max problems.

Contribution

It proposes a novel mirror-free mirror prox method, broadening the applicability of proximal algorithms without requiring a mirror map.

Findings

Convergence analysis under relative smoothness and Lipschitzness.

Extension to strongly monotone problems and min-max optimization.

Generalization of Bregman divergence using potential operators.

Abstract

We present a \emph{mirror-free} mirror prox (MFMP) algorithm, which extends the classic approach of Nemirovski (2004) to allow for proximal-like updates without the explicit need for a mirror map. We further analyze the convergence of our method under suitable notions of relative smoothness and relative Lipschitzness, for which we introduce a relaxation of the standard Bregman divergence in terms of more general potential operators. Finally, we show how a strongly monotone variant of our method allows us to solve regularized Taylor-expansion subproblems that appear in both second- and third-order smooth min-max optimization.