Saddle Point Optimization with Approximate Minimization Oracle

TL;DR

This paper proposes a novel saddle point optimization method based on approximate minimization oracles, demonstrating linear convergence under certain conditions and introducing adaptive, derivative-free algorithms with practical numerical validation.

Contribution

It introduces an alternative saddle point optimization approach relying on approximate minimization oracles, with theoretical convergence guarantees and adaptive algorithms.

Findings

Linear convergence on locally strong convex-concave functions

Effective eta adaptation improves practical performance

Numerical experiments validate theoretical results

Abstract

A major approach to saddle point optimization $\min_x\max_y f(x, y)$ is a gradient based approach as is popularized by generative adversarial networks (GANs). In contrast, we analyze an alternative approach relying only on an oracle that solves a minimization problem approximately. Our approach locates approximate solutions $x'$ and $y'$ to $\min_{x'}f(x', y)$ and $\max_{y'}f(x, y')$ at a given point $(x, y)$ and updates $(x, y)$ toward these approximate solutions $(x', y')$ with a learning rate $\eta$. On locally strong convex--concave smooth functions, we derive conditions on $\eta$ to exhibit linear convergence to a local saddle point, which reveals a possible shortcoming of recently developed robust adversarial reinforcement learning algorithms. We develop a heuristic approach to adapt $\eta$ derivative-free and implement zero-order and first-order minimization algorithms. Numerical experiments are conducted to show the tightness of the theoretical results as well as the usefulness of the $\eta$ adaptation mechanism.