A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach

TL;DR

This paper presents a unified analysis of Extra-gradient and Optimistic Gradient methods for saddle point problems, using a proximal point approach to unify and extend their theoretical understanding.

Contribution

It introduces a novel proximal point framework that unifies the analysis of EG and OGDA methods and generalizes results for various parameters.

Findings

Unified analysis of EG and OGDA as proximal point approximations

Extension of OGDA results to broader parameter ranges

New insights into convergence in bilinear and strongly convex-strongly concave settings

Abstract

In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algorithms admit a unified analysis as approximations of the classical proximal point method for solving saddle point problems. This viewpoint enables us to develop a new framework for analyzing EG and OGDA for bilinear and strongly convex-strongly concave settings. Moreover, we use the proximal point approximation interpretation to generalize the results for OGDA for a wide range of parameters.