Convergence Rate of $\mathcal{O}(1/k)$ for Optimistic Gradient and Extra-gradient Methods in Smooth Convex-Concave Saddle Point Problems

TL;DR

This paper establishes an $ ext{O}(1/k)$ convergence rate for the optimistic gradient descent-ascent and extra-gradient methods in smooth convex-concave saddle point problems, providing new theoretical insights.

Contribution

It offers the first convergence rate estimate for OGDA in the general convex-concave setting and simplifies the analysis of EG without compactness assumptions.

Findings

Both OGDA and EG have bounded iterates.

Primal-dual gap converges at a rate of $ ext{O}(1/k)$ for averaged iterates.

Provides a unified interpretation of OGDA and EG as approximate proximal point methods.

Abstract

We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extra-gradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method. This is similar to the approach taken in [Nemirovski, 2004] which analyzes EG as an approximation of the `conceptual mirror prox'. In this paper, we highlight how gradients used in OGDA and EG try to approximate the gradient of the Proximal Point method. We then exploit this interpretation to show that both algorithms produce iterates that remain within a bounded set. We further show that the primal dual gap of the averaged iterates generated by both of these algorithms converge with a rate of $\mathcal{O}(1/k)$. Our theoretical analysis is of interest as it provides a the first convergence rate estimate for OGDA in the general convex-concave setting. Moreover, it provides a simple convergence analysis for the EG algorithm in terms of function value without using compactness assumption.

Equations

$$\underset{\mathbf{x}\in {\mathbb{R}}^m}{\min }\underset{\mathbf{y}\in {\mathbb{R}}^n}{\max }f(\mathbf{x},\mathbf{y}),$$

$$f({\mathbf{x}}^{\ast },\mathbf{y})\le f({\mathbf{x}}^{\ast },{\mathbf{y}}^{\ast })\le f(\mathbf{x},{\mathbf{y}}^{\ast })$$

$$||\nabla \varphi (\mathbf{x})-\nabla \varphi (\hat{\mathbf{x}})||\le L||\mathbf{x}-\hat{\mathbf{x}}||.$$

$$\varphi (\hat{\mathbf{x}})\ge \varphi (\mathbf{x})+\nabla \varphi (\mathbf{x}{)}^T(\hat{\mathbf{x}}-\mathbf{x}).$$

$$f({\mathbf{x}}^{\ast },\mathbf{y})\le f({\mathbf{x}}^{\ast },{\mathbf{y}}^{\ast })\le f(\mathbf{x},{\mathbf{y}}^{\ast }).$$

$$\Vert {\nabla }_{\mathbf{x}}f({\mathbf{x}}_1,\mathbf{y})-{\nabla }_{\mathbf{x}}f({\mathbf{x}}_2,\mathbf{y})\Vert$$

$$\Vert {\nabla }_{\mathbf{x}}f(\mathbf{x},{\mathbf{y}}_1)-{\nabla }_{\mathbf{x}}f(\mathbf{x},{\mathbf{y}}_2)\Vert$$

$$\Vert {\nabla }_{\mathbf{y}}f(\mathbf{x},{\mathbf{y}}_1)-{\nabla }_{\mathbf{y}}f(\mathbf{x},{\mathbf{y}}_2)|$$

$$\Vert {\nabla }_{\mathbf{y}}f({\mathbf{x}}_1,\mathbf{y})-{\nabla }_{\mathbf{y}}f({\mathbf{x}}_2,\mathbf{y})|$$

$${\mathcal{Z}}^{\ast }:=\{[\mathbf{x};\mathbf{y}]\in {\mathbb{R}}^{n+m}:(\mathbf{x},\mathbf{y})\text{ is a saddle point of Problem }\text{eqref}mai{n}_{p}rob\},$$

$${\hat{\mathbf{x}}}_k=\frac{1}{k}\sum _{i=1}^k{\mathbf{x}}_i,{\hat{\mathbf{y}}}_k=\frac{1}{k}\sum _{i=1}^k{\mathbf{y}}_i.$$

$$F(\mathbf{z})=[{\nabla }_{\mathbf{x}}f(\mathbf{x},\mathbf{y});-{\nabla }_{\mathbf{y}}f(\mathbf{x},\mathbf{y})].$$

$$⟨F({\mathbf{z}}_1)-F({\mathbf{z}}_2),{\mathbf{z}}_1-{\mathbf{z}}_2⟩\ge 0.$$

$$\Vert F({\mathbf{z}}_1)-F({\mathbf{z}}_2)\Vert \le L\Vert {\mathbf{z}}_1-{\mathbf{z}}_2\Vert .$$

$${\nabla }_{\mathbf{x}}f({\mathbf{x}}^{\ast },{\mathbf{y}}^{\ast })=0{\nabla }_{\mathbf{y}}f({\mathbf{x}}^{\ast },{\mathbf{y}}^{\ast })$$

$$f({\hat{\mathbf{x}}}_N,\mathbf{y})-f(\mathbf{x},{\hat{\mathbf{y}}}_N)\le \frac{1}{N}\sum _{k=1}^{N}F({\mathbf{z}}_k{)}^{\top }({\mathbf{z}}_k-\mathbf{z})$$

$$\frac{1}{N}\sum _{k=1}^{N}F({\mathbf{z}}_k{)}^{\top }({\mathbf{z}}_k-\mathbf{z})$$

$$\ge \frac{1}{N}\sum _{k=1}^N[f({\mathbf{x}}_k,{\mathbf{y}}_k)-f(\mathbf{x},{\mathbf{y}}_k)+f({\mathbf{x}}_k,\mathbf{y})-f({\mathbf{x}}_k,{\mathbf{y}}_k)]$$

$$=\frac{1}{N}\sum _{k=1}^N[f({\mathbf{x}}_k,\mathbf{y})-f(\mathbf{x},{\mathbf{y}}_k)],$$

$$\frac{1}{N}\sum _{k=1}^{N}f({\mathbf{x}}_k,\mathbf{y})\ge f({\hat{\mathbf{x}}}_N,\mathbf{y}),\frac{1}{N}\sum _{k=1}^{N}f(\mathbf{x},{\mathbf{y}}_k)\le f(\mathbf{x},{\hat{\mathbf{y}}}_N).$$

$$\frac{1}{N}\sum _{k=1}^{N}F({\mathbf{z}}_k{)}^{\top }({\mathbf{z}}_k-\mathbf{z})$$

$$\underset{\mathbf{x}\in {\mathbb{R}}^m}{\min }\underset{\mathbf{y}\in {\mathbb{R}}^n}{\max }\left\{f(\mathbf{x},\mathbf{y})+\frac{1}{2\eta }\Vert \mathbf{x}-{\mathbf{x}}_k{\Vert }^2-\frac{1}{2\eta }\Vert \mathbf{y}-{\mathbf{y}}_k{\Vert }^2\right\}.$$

$${\mathbf{x}}_{k+1}$$

$${\mathbf{y}}_{k+1}$$

$${\mathbf{x}}_{k+1}$$

$${\mathbf{y}}_{k+1}$$

$$\left|f({\hat{\mathbf{x}}}_k,{\hat{\mathbf{y}}}_k)-f({\mathbf{x}}^{\ast },{\mathbf{y}}^{\ast })\right|\le \frac{\Vert {\mathbf{x}}_0-{\mathbf{x}}^{\ast }{\Vert }^2+\Vert {\mathbf{y}}_0-{\mathbf{y}}^{\ast }{\Vert }^2}{\eta k}.$$

$${\mathbf{z}}_{k+1}={\mathbf{z}}_k-\eta F({\mathbf{z}}_{k+1}),$$

$${\mathbf{z}}_{k+1}={\mathbf{z}}_k-\eta F({\mathbf{z}}_{k+1})+{\mathbf{\epsilon }}_k,$$

$$F({\mathbf{z}}_{k+1}{)}^{\top }({\mathbf{z}}_{k+1}-\mathbf{z})$$

$$=\frac{1}{2\eta }\Vert {\mathbf{z}}_k-\mathbf{z}{\Vert }^2-\frac{1}{2\eta }\Vert {\mathbf{z}}_{k+1}-\mathbf{z}{\Vert }^2-\frac{1}{2\eta }\Vert {\mathbf{z}}_{k+1}-{\mathbf{z}}_k{\Vert }^2+\frac{1}{\eta }{{\mathbf{\epsilon }}_k}^{\top }({\mathbf{z}}_{k+1}-\mathbf{z}).$$

$$\Vert {\mathbf{z}}_{k+1}-\mathbf{z}{\Vert }^2=\Vert {\mathbf{z}}_k-\mathbf{z}{\Vert }^2-2\eta ({\mathbf{z}}_k-\mathbf{z}{)}^{\top }F({\mathbf{z}}_{k+1})$$

$$+2{{\mathbf{\epsilon }}_k}^{\top }({\mathbf{z}}_k-\mathbf{z}-\eta F({\mathbf{z}}_{k+1})).$$

$$\Vert {\mathbf{z}}_{k+1}-\mathbf{z}{\Vert }^2$$

$$+{\eta }^2\Vert F({\mathbf{z}}_{k+1}){\Vert }^2+\Vert {\mathbf{\epsilon }}_k{\Vert }^2+2{{\mathbf{\epsilon }}_k}^T({\mathbf{z}}_k-\mathbf{z}-\eta F({\mathbf{z}}_{k+1})).$$

Full text

1 Introduction

Given a function $f:{\mathbb{R}}^{m}\times{\mathbb{R}}^{n}\to{\mathbb{R}}$, we consider finding a saddle point of the problem

[TABLE]

where a saddle point of Problem (1) is defined as a pair $({\mathbf{x}}^{*},{\mathbf{y}}^{*})\in{\mathbb{R}}^{m}\times{\mathbb{R}}^{n}$ that satisfies

[TABLE]

for all ${\mathbf{x}}\in{\mathbb{R}}^{m},{\mathbf{y}}\in{\mathbb{R}}^{n}$. Throughout the paper, we assume that the function $f({\mathbf{x}},{\mathbf{y}})$ is convex-concave, i.e., for any ${\mathbf{y}}\in\mathbb{R}^{n}$, the function $f({\mathbf{x}},{\mathbf{y}})$ is a convex function of ${\mathbf{x}}$ and for any ${\mathbf{x}}\in\mathbb{R}^{m}$, the function $f({\mathbf{x}},{\mathbf{y}})$ is a concave function of ${\mathbf{y}}$. This formulation arises in several areas, including zero-sum games [Basar & Olsder, 1999], robust optimization [Ben-Tal et al., 2009], robust control [Hast et al., 2013] and more recently in machine learning in the context of Generative Adversarial Networks (GANs) (see [Goodfellow et al., 2014] for an introduction to GANs and [Arjovsky et al., 2017] for the formulation of Wasserstein GANs).

Our goal in this paper is to analyze the convergence rate of some discrete-time gradient based optimization algorithms for finding a saddle point of Problem (1) in the convex-concave case. In particular, we focus on Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods because of their widespread use for training GANs (see [Daskalakis et al., 2018; Liang & Stokes, 2019]). EG method is a classical algorithm for solving saddle point problems introduced by Korpelevich [1976]. Its linear rate of convergence for smooth and strongly convex-strongly concave functions $f({\mathbf{x}},{\mathbf{y}})$ 111Note that when we state that $f({\mathbf{x}},{\mathbf{y}})$ is strongly convex-strongly concave, it means that $f(\cdot,{\mathbf{y}})$ is strongly convex for all ${\mathbf{y}}\in{\mathbb{R}}^{n}$ and $f({\mathbf{x}},\cdot)$ is strongly concave for all ${\mathbf{x}}\in{\mathbb{R}}^{m}$. and bilinear functions, i.e., $f({\mathbf{x}},{\mathbf{y}})={\mathbf{x}}^{\top}{\mathbf{A}}{\mathbf{y}}$ (where ${\mathbf{A}}$ is a square, full rank matrix), was established in Korpelevich [1976] as well as the variational inequality literature (see [Tseng, 1995] and [Facchinei & Pang, 2007]). Its $\mathcal{O}(1/k)$ convergence rate for the constrained convex-concave setting was first established by Nemirovski [2004] under the assumption that the feasible set is convex and compact.222The result in [Nemirovski, 2004] shows a $\mathcal{O}(1/k)$ convergence rate for the mirror-prox algorithm which specializes to the EG method for the Euclidean case. Monteiro & Svaiter [2010] established a similar $\mathcal{O}(1/k)$ convergence rate for EG without assuming compactness of the feasible set by using a new termination criterion that relies on enlargement of the operator of the VI reformulation of the saddle point problem defined in [Burachik et al., 1997]. OGDA was introduced by Popov [1980], as a variant of the Extragradient method, and has gained popularity recently due to its performance in training GANs (see [Daskalakis et al., 2018]). To the best of our knowledge, iteration complexity of OGDA for the convex-concave case has not been studied before.

In this paper, we provide a unified convergence analysis for establishing a sublinear convergence rate of $\mathcal{O}(1/k)$ in terms of the function value difference of the averaged iterates and a saddle point for both OGDA and EG for convex-concave saddle point problems. Our analysis holds for unconstrained problems and does not require boundedness of the feasible set, and it establishes rate results using the function value differences as used in [Nemirovski, 2004] (suitably redefined for an unconstrained feasible set, see Section 5). Therefore, we get convergence of the EG method in unconstrained spaces without using the modified termination (error) criterion proposed in [Monteiro & Svaiter, 2010]. The key idea of our approach is to view both OGDA and EG iterates as approximations of the iterates of the proximal point method that was first introduced by Martinet [1970] and later studied by Rockafellar [1976]. We would like to add that the idea of interpreting EG as an approximation of the Proximal Point method was first studied in [Nemirovski, 2004]. He considers the conceptual mirror prox, which is similar to the proximal point method, and shows that the mirror prox algorithm (of which EG is a special case) provides a good implementable approximation to this method. Further, Monteiro & Svaiter [2010] use a similar interpretation and propose the Hybrid Proximal Extragradient method to establish the convergence of EG in unbounded settings using a different convergence criteria. More recently, Mokhtari et al. [2020] study both OGDA and EG as approximations of proximal point method and analyze these algorithms for bilinear and strongly convex-strongly concave problems.

More specifically, we first consider a proximal point method with error and establish some key properties of its iterates. We then focus on OGDA as an approximation of proximal point method and use this connection to show that the iterates of OGDA remain in a compact set. We incorporate this result to prove a sublinear convergence rate of $\mathcal{O}(1/k)$ for the primal-dual gap of the averaged iterates generated by the OGDA update. We next consider EG where two gradient pairs are used in each iteration, one to compute a midpoint and other to find the new iterate using the gradient of the midpoint. Our first step again is to show boundedness of the iterates generated by EG. We then approximate the evolution of the midpoints using a proximal point method and use this approximation to establish $\mathcal{O}(1/k)$ convergence rate for the function value of the averaged iterates generated by EG. As the convergence results of EG have already been established in papers including [Nemirovski, 2004] and Monteiro & Svaiter [2010], we relegate the proofs of Lemmas and Theorems corresponding to EG to the Appendix.

Related Work

Several recent papers have studied the convergence rate of OGDA and EG for the case when the objective function is bilinear or strongly convex-strongly concave. Daskalakis et al. [2018] showed the convergence of the OGDA iterates to a neighborhood of the solution when the objective function is bilinear. Liang & Stokes [2019] used a dynamical system approach to prove the linear convergence of the OGDA method for the special case when $f({\mathbf{x}},{\mathbf{y}})={\mathbf{x}}^{\top}{\mathbf{A}}{\mathbf{y}}$ and the matrix ${\mathbf{A}}$ is square and full rank. They also presented a linear convergence rate of the vanilla Gradient Ascent Descent (GDA) method when the objective function $f({\mathbf{x}},{\mathbf{y}})$ is strongly convex-strongly concave. Gidel et al. [2018] considered a variant of the EG method, relating it to OGDA updates, and showed the linear convergence of the corresponding EG iterates in the case where $f({\mathbf{x}},{\mathbf{y}})$ is strongly convex-strongly concave (though without showing the convergence rate for the OGDA iterates). Optimistic gradient methods have also been studied in the context of convex online learning [Chiang et al., 2012; Rakhlin & Sridharan, 2013a, b].

Nedić & Ozdaglar [2009] analyzed the (sub)Gradient Descent Ascent (GDA) algorithm for convex-concave saddle point problems when the (sub)gradients are bounded over the constraint set, showing a convergence rate of $\mathcal{O}(1/\sqrt{k})$ in terms of the function value difference of the averaged iterates and a saddle point.

Chambolle & Pock [2011] focused on a particular case of the saddle point problem where the coupling term in the objective function is bilinear, i.e., $f({\mathbf{x}},{\mathbf{y}})=G({\mathbf{x}})+{\mathbf{x}}^{\top}{\mathbf{K}}{\mathbf{y}}-H({\mathbf{y}})$ with $G$ and $H$ convex functions. They proposed a proximal point based algorithm which converges at a rate $\mathcal{O}(1/k)$ and further showed linear convergence when the functions $G$ and $H$ are strongly convex. Chen et al. [2014] proposed an accelerated variant of this algorithm when $G$ is smooth and showed an optimal rate of $\mathcal{(}\frac{L_{G}}{k^{2}}+\frac{L_{K}}{k})$, where $L_{G}$ and $L_{K}$ are the smoothness parameters of $G$ and the norm of the linear operator $K$ respectively. When the functions $G$ and $H$ are strongly convex, primal-dual gradient-type methods converge linearly, as shown in Chen & Rockafellar [1997]; Bauschke et al. [2011]. Further, Du & Hu [2019] showed that GDA achieves a linear convergence rate in this linearly coupled setting when $G$ is convex and $H$ is strongly convex.

For the case that $f({\mathbf{x}},{\mathbf{y}})$ is strongly concave with respect to ${\mathbf{y}}$, but possibly nonconvex with respect to ${\mathbf{x}}$, Sanjabi et al. [2018] provided convergence to a first-order stationary point using an algorithm that requires running multiple updates with respect to ${\mathbf{y}}$ at each step.

**Notation. ** Lowercase boldface ${\mathbf{v}}$ denotes a vector and uppercase boldface ${\mathbf{A}}$ denotes a matrix. We use $\|{\mathbf{v}}\|$ to denote the Euclidean norm of vector ${\mathbf{v}}$. Given a multi-input function $f({\mathbf{x}},{\mathbf{y}})$, its gradient with respect to ${\mathbf{x}}$ and ${\mathbf{y}}$ at points $({\mathbf{x}}_{0},{\mathbf{y}}_{0})$ are denoted by $\nabla_{\mathbf{x}}f({\mathbf{x}}_{0},{\mathbf{y}}_{0})$ and $\nabla_{\mathbf{y}}f({\mathbf{x}}_{0},{\mathbf{y}}_{0})$, respectively. We refer to the largest and smallest eigenvalues of a matrix ${\mathbf{A}}$ by $\lambda_{\max}({\mathbf{A}})$ and $\lambda_{\min}({\mathbf{A}})$, respectively.

2 Preliminaries

In this section we present properties and notations used in our results.

Definition 1**.**

A function $\phi:{\mathbb{R}}^{n}\to{\mathbb{R}}$ is $L$-smooth if it has $L$-Lipschitz continuous gradients on ${\mathbb{R}}^{n}$, i.e., for any ${\mathbf{x}},{\hat{\mathbf{x}}}\in{\mathbb{R}}^{n}$, we have

[TABLE]

Definition 2**.**

A continuously differentiable function $\phi:{\mathbb{R}}^{n}\to{\mathbb{R}}$ is convex on $\mathbb{R}^{n}$ if for any ${\mathbf{x}},{\hat{\mathbf{x}}}\in{\mathbb{R}}^{n}$, we have

[TABLE]

Further, $\phi({\mathbf{x}})$ is concave if $-\phi({\mathbf{x}})$ is convex.

Definition 3**.**

The pair $({\mathbf{x}}^{*},{\mathbf{y}}^{*})$ is a saddle point of a convex-concave function $f({\mathbf{x}},{\mathbf{y}})$, if for any ${\mathbf{x}}\in{\mathbb{R}}^{n}$ and ${\mathbf{y}}\in{\mathbb{R}}^{m}$, we have

[TABLE]

Throughout the paper, we will assume that the following conditions are satisfied.

Assumption 1**.**

The function $f({\mathbf{x}},{\mathbf{y}})$ is continuously differentiable in ${\mathbf{x}}$ and ${\mathbf{y}}$. Further, for any ${\mathbf{y}}\in\mathbb{R}^{n}$, the function $f({\mathbf{x}},{\mathbf{y}})$ is a convex function of ${\mathbf{x}}$ and for any ${\mathbf{x}}\in\mathbb{R}^{m}$, the function $f({\mathbf{x}},{\mathbf{y}})$ is a concave function of ${\mathbf{y}}$.

Assumption 2**.**

The gradient $\nabla_{{\mathbf{x}}}f({\mathbf{x}},{\mathbf{y}})$, is $L_{xx}$-Lipschitz with respect to ${\mathbf{x}}$ and $L_{xy}$-Lipschitz with respect to ${\mathbf{y}}$ and the gradient $\nabla_{{\mathbf{y}}}f({\mathbf{x}},{\mathbf{y}})$, is $L_{yy}$-Lipschitz with respect to ${\mathbf{y}}$ and $L_{yx}$-Lipschitz with respect to ${\mathbf{x}}$, i.e.,

[TABLE]

We define $L:=2\times\max\{L_{xx},L_{xy},L_{yx},L_{yy}\}$. 333 In this definition we need an additional factor of 2 because in the analysis we use $L$ as the Lipschitz continuity of the operator $F(\cdot)=[\nabla_{\mathbf{x}}f(\cdot);-\nabla_{\mathbf{y}}f(\cdot)]$.

Assumption 3**.**

The solution set $\mathcal{Z}^{*}$ defined as

[TABLE]

is nonempty.

In the following sections, we present and analyze three different iterative algorithms for solving the saddle point problem introduced in (1). The $k^{th}$ iterates of these algorithms are denoted by $({\mathbf{x}}_{k},{\mathbf{y}}_{k})$. We denote the averaged (ergodic) iterates by $\hat{{\mathbf{x}}}_{k},\hat{{\mathbf{y}}}_{k}$, defined as follows:

[TABLE]

In our convergence analysis, we use a variational inequality approach in which we define the vector ${\mathbf{z}}=[{\mathbf{x}};{\mathbf{y}}]\in{\mathbb{R}}^{n+m}$ as our decision variable and define the operator $F:{\mathbb{R}}^{m+n}\to{\mathbb{R}}^{m+n}$ as

[TABLE]

In the following lemma we characterize the properties of operator $F$ in (4) when the conditions in Assumptions 1 and 2 are satisfied. We would like to emphasize that the following lemma is well-known – see, e.g., Nemirovski [2004] – and we state it for completeness.

Lemma 1**.**

*Let $F(\cdot)$ be defined as in Equation (4). Suppose Assumptions 1 and 2 hold. Then

(a) $F$ is a monotone operator, i.e., for any ${\mathbf{z}}_{1},{\mathbf{z}}_{2}\in{\mathbb{R}}^{m+n}$, we have*

[TABLE]

(b) $F$ is an $L$-Lipschitz continuous operator, i.e., for any ${\mathbf{z}}_{1},{\mathbf{z}}_{2}\in{\mathbb{R}}^{m+n}$, we have

[TABLE]

(c) For all ${\mathbf{z}}^{*}\in\mathcal{Z}^{*}$, we have $F({\mathbf{z}}^{*})=0$.

According to Lemma 1, when $f$ is convex-concave and smooth, the operator $F$ defined in (4) is monotone and Lipschitz. The third result in Lemma 1 shows that any saddle point of problem (1) satisfies the first-order optimality condition, i.e $\text{for all}\ ({\mathbf{x}}^{*},{\mathbf{y}}^{*})\in\mathcal{Z}^{*}$, we have:

[TABLE]

Before presenting our main results, we state the following well known result (see for example Nemirovski [2004]) which will be used later in the analysis of OGDA and EG. We present the proof here for completeness.

Proposition 1**.**

Recall the definition of the operator $F(\cdot)$ in (4) and the points ${\hat{\mathbf{x}}}_{k},{\hat{\mathbf{y}}}_{k}$ in (3). Suppose Assumptions 1 and 3 hold. Then for any ${\mathbf{z}}=[{\mathbf{x}};{\mathbf{y}}]\in{\mathbb{R}}^{m+n}$, we have

[TABLE]

Proof.

Using the definition of the operator $F$, we can write

[TABLE]

where the inequality holds due to the fact that $f$ is convex-concave. Using convexity of $f$ with respect to ${\mathbf{x}}$ and concavity of $f$ with respect to ${\mathbf{y}}$, we have

[TABLE]

Combining inequalities (7) and (8) yields

[TABLE]

completing the proof. ∎

3 Proximal point method with error

One of the classical algorithms studied for solving the saddle point problem in (1) is the Proximal Point (PP) method, introduced in Martinet [1970] and studied in Rockafellar [1976]. The PP method generates the iterate $\{{\mathbf{x}}_{k+1},{\mathbf{y}}_{k+1}\}$ which is defined as the unique solution to the saddle point problem444Again $\{{\mathbf{x}}_{k+1},{\mathbf{y}}_{k+1}\}$ is unique since the objective function of problem (9) is strongly convex in ${\mathbf{x}}$ and strongly concave in ${\mathbf{y}}$

[TABLE]

It can be verified that if the pair $\{{\mathbf{x}}_{k+1},{\mathbf{y}}_{k+1}\}$ is the solution of problem (9), then ${\mathbf{x}}_{k+1}$ and ${\mathbf{y}}_{k+1}$ satisfy

[TABLE]

Using the optimality conditions of the updates in (10) and (11) (which are necessary and sufficient since the problems in (10) and (11) are strongly convex and strongly concave, respectively), the update of the PP method for the saddle point problem in (1) can be written as

[TABLE]

It is well-known that the proximal point method achieves a sublinear rate of $\mathcal{O}(1/k)$ when $k$ is the number of iterations for convex minimization and for solving monotone variational inequalities (see Güler [1991, 1992]; Bruck Jr [1977]; Teboulle [1997]; Nemirovski [2004]). Note that Nemirovski [2004] in fact analyzed the conceptual mirror prox (the proximal point method) as a building block to analyze the mirror-prox algorithm. For completeness, we present the convergence rate of the proximal point method for convex-concave saddle point problems in the following theorem (see Appendix A for the proof).

Theorem 1**.**

Suppose Assumption 1 holds. Let $\{{\mathbf{x}}_{k},{\mathbf{y}}_{k}\}$ be the iterates generated by the updates in (3). Consider the definition of the averaged iterates ${\hat{\mathbf{x}}}_{k},{\hat{\mathbf{y}}}_{k}$ in (3). Then for all $k\geq 1$, we have

[TABLE]

The result in Theorem 1 shows that by following the update of proximal point method the gap between the function value for the averaged iterates $(\hat{{\mathbf{x}}}_{k},\hat{{\mathbf{y}}}_{k})$ and the function value for a saddle point $({\mathbf{x}}^{*},{\mathbf{y}}^{*})$ of the problem (1) approaches zero at a sublinear rate of $\mathcal{O}(1/k)$.

Our goal is to provide similar convergence rate estimates for OGDA and EG using the fact that these two methods can be interpreted as approximate versions of the proximal point method. To do so, let us first rewrite the update of the proximal point method given in (3) as

[TABLE]

where ${\mathbf{z}}=[{\mathbf{x}};{\mathbf{y}}]\in\mathbb{R}^{m+n}$ and the operator $F$ is defined in (4). In the following proposition, we establish a relation for the iterates of a proximal point method with error. This relation will be used later for our analysis of OGDA and EG methods.

Proposition 2**.**

Consider the sequence of iterates $\{{\mathbf{z}}_{k}\}\in{\mathbb{R}}^{n+m}$ generated by the following update

[TABLE]

where $F:{\mathbb{R}}^{n+m}\to{\mathbb{R}}^{n+m}$ is a monotone and Lipschitz continuous operator, $\boldsymbol{\varepsilon}_{k}\in{\mathbb{R}}^{n+m}$ is an arbitrary vector, and $\eta$ is a positive constant. Then for any ${\mathbf{z}}\in{\mathbb{R}}^{n+m}$ and for each $k\geq 1$ we have

[TABLE]

Proof.

According to the update in (15), we can show that for any ${\mathbf{z}}\in{\mathbb{R}}^{m+n}$ we have

[TABLE]

We add and subtract the inner product $2\eta{\mathbf{z}}_{k+1}^{\top}F({\mathbf{z}}_{k+1})$ to the right hand side and regroup the terms to obtain

[TABLE]

Replacing $F({\mathbf{z}}_{k+1})$ with $(1/\eta)(-{\mathbf{z}}_{k+1}+{\mathbf{z}}_{k}+\boldsymbol{\varepsilon}_{k})$, we obtain

[TABLE]

On rearranging the terms, we obtain the following inequality:

[TABLE]

and the proof is complete. ∎

4 Optimistic Gradient Descent Ascent

In this section, we focus on analyzing the performance of optimistic gradient descent ascent (OGDA) for finding a saddle point of a general smooth convex-concave function. It has been shown that the OGDA method achieves the same iteration complexity as the proximal point method for both strongly convex-strongly concave and bilinear problems; see Liang & Stokes [2019], Gidel et al. [2018], Mokhtari et al. [2020]. However, its iteration complexity for a general smooth convex-concave case has not been established to the best of our knowledge. In this section, we show that the function value of the averaged iterate generated by the OGDA method converges to the function value at a saddle point at a rate of $\mathcal{O}(1/k)$, which matches the convergence rate of the proximal point method shown in Theorem 1.

Given a stepsize $\eta>0$, the OGDA method updates the iterates ${\mathbf{x}}_{k}$ and ${\mathbf{y}}_{k}$ for each $k\geq 0$ as

[TABLE]

with the initial conditions ${\mathbf{x}}_{0}={\mathbf{x}}_{-1}$ and ${\mathbf{y}}_{0}={\mathbf{y}}_{-1}$. The main difference between the updates of OGDA in (4) and the gradient descent ascent (GDA) method is in the additional “momentum” terms $-\eta(\nabla_{{\mathbf{x}}}f\left({\mathbf{x}}_{k},{\mathbf{y}}_{k}\right)-\nabla_{{\mathbf{x}}}f\left({\mathbf{x}}_{k-1},{\mathbf{y}}_{k-1}\right))$ and $\eta(\nabla_{{\mathbf{y}}}f\left({\mathbf{x}}_{k},{\mathbf{y}}_{k}\right)-\nabla_{{\mathbf{y}}}f\left({\mathbf{x}}_{k-1},{\mathbf{y}}_{k-1}\right))$. This additional term makes the update of OGDA a better approximation to the update of the proximal point method compared to the update of the GDA; for more details we refer readers to Proposition 1 in Mokhtari et al. [2020].

To establish the convergence rate of OGDA for convex-concave problems, we first illustrate the connection between the updates of proximal point method and OGDA. Note that using the definitions of the vector ${\mathbf{z}}=[{\mathbf{x}};{\mathbf{y}}]\in{\mathbb{R}}^{n+m}$ and the operator $F({\mathbf{z}})=[\nabla_{{\mathbf{x}}}f({\mathbf{x}},{\mathbf{y}});-\nabla_{{\mathbf{y}}}f({\mathbf{x}},{\mathbf{y}})]\in{\mathbb{R}}^{n+m}$, we can rewrite the update of the OGDA algorithm at iteration $k$ as

[TABLE]

Considering this expression, we can also write the update of OGDA as an approximation of the proximal point update, i.e.,

[TABLE]

where the error vector $\boldsymbol{\varepsilon}_{k}$ is given by

[TABLE]

To derive the convergence rate of OGDA for the unconstrained problem in (1), we first use the result in Proposition 2 to derive a result for the specific case of OGDA updates. We then show that the iterates generated by the OGDA method remain in a bounded set. This is done in the following lemma (Note that boundedness of OGDA iterates can be deduced from [Popov, 1980], whereas a result similar to Lemma 2(b) was shown in a recent independent paper by Malitsky & Tam [2018]).

Lemma 2**.**

*Let $\{{\mathbf{z}}_{k}\}$ be the iterates generated by the optimistic gradient descent ascent (OGDA) method introduced in (22) with the initial conditions ${\mathbf{x}}_{0}={\mathbf{x}}_{-1}$ and ${\mathbf{y}}_{0}={\mathbf{y}}_{-1}$ (i.e. ${\mathbf{z}}_{0}={\mathbf{z}}_{-1}$). If Assumptions 1, 2, and 3 hold and the stepsize $\eta$ satisfies the condition $0<\eta\leq\frac{1}{2L}$, then:

(a) The iterates $\{{\mathbf{z}}_{k}\}$ satisfy the following relation:*

[TABLE]

(b) The iterates $\{{\mathbf{z}}_{k}\}$ stay within the compact set ${\mathcal{D}}$ defined as

[TABLE]

where $({\mathbf{x}}^{*},{\mathbf{y}}^{*})={\mathbf{z}}^{*}\in\mathcal{Z}^{*}$ is a saddle point of the problem defined in (1).

Proof.

Since OGDA iterates satisfy Equation (23) with the error vector $\boldsymbol{\varepsilon}_{k}$ given in Equation (24), using Proposition 2 with this error vector $\boldsymbol{\varepsilon}_{k}$ leads to

[TABLE]

We add and subtract the inner product $(F({\mathbf{z}}_{k})-F({\mathbf{z}}_{k-1}))^{\top}({\mathbf{z}}_{k}-{\mathbf{z}})$ to the right hand side of the preceding relation to obtain

[TABLE]

Note that $(F({\mathbf{z}}_{k})-F({\mathbf{z}}_{k-1}))^{\top}({\mathbf{z}}_{k}-{\mathbf{z}}_{k+1})$ can be upper bounded by

[TABLE]

where the second inequality holds due to Lipschitz continuity of the operator $F$ (Lemma 1(b)) and the last inequality holds due to Young’s inequality.555We use the following form of Young’s inequality throughout the paper:

${\mathbf{a}}^{\top}{\mathbf{b}}\leq\frac{\|{\mathbf{a}}\|^{2}}{2}+\frac{\|{\mathbf{b}}\|^{2}}{2}$

Replacing $(F({\mathbf{z}}_{k})-F({\mathbf{z}}_{k-1}))^{\top}({\mathbf{z}}_{k}-{\mathbf{z}}_{k+1})$ in (4) by its upper bound in (4) yields

[TABLE]

where the second inequality follows as $\eta\leq 1/2L$ and therefore $-\frac{1}{2\eta}\|{\mathbf{z}}_{k+1}-{\mathbf{z}}_{k}\|^{2}\leq-{L}\|{\mathbf{z}}_{k+1}-{\mathbf{z}}_{k}\|^{2}$. This completes the proof of Part (a) of the lemma. Now, taking the sum of the preceding relation from $k=0,\cdots,N-1$, we obtain

[TABLE]

Now set ${\mathbf{z}}={\mathbf{z}}^{*}$, where ${\mathbf{z}}^{*}\in\mathcal{Z}^{*}$, to obtain

[TABLE]

Note that each term of the summand in the sum in the left is nonnegative due to monotonicity of $F$ and therefore the sum is also nonnegative. Further, we know that ${\mathbf{z}}_{0}={\mathbf{z}}_{-1}$. Using these observations we can write

[TABLE]

Using Lipschitz continuity of the operator $F(\cdot)$ (Lemma 1(b)) and Young’s inequality in the preceding relation, we have

[TABLE]

Regrouping the terms gives us

[TABLE]

Using the condition $\eta\leq 1/2L$, it follows that for any iterate $N$ we have

[TABLE]

and the claim in Part (b) follows. ∎

According to Lemma 2, the sequence of iterates $\{{\mathbf{x}}_{k},{\mathbf{y}}_{k}\}$ generated by OGDA method stays within a closed and bounded convex set. We use this result to prove a sublinear convergence rate of $\mathcal{O}(1/k)$ for the function value of the averaged iterates generated by OGDA to the function value at a saddle point, for smooth and convex-concave functions in the following theorem.

Theorem 2**.**

*Suppose Assumptions 1, 2 and 3 hold. Let $\{{\mathbf{x}}_{k},{\mathbf{y}}_{k}\}$ be the iterates generated by the OGDA updates in (4). Let the initial conditions satisfy ${\mathbf{x}}_{0}={\mathbf{x}}_{-1}$ and ${\mathbf{y}}_{0}={\mathbf{y}}_{-1}$. Consider the definition of the averaged iterates ${\hat{\mathbf{x}}}_{N},{\hat{\mathbf{y}}}_{N}$ in (3) and the compact convex set ${\mathcal{D}}$ in (26). If the stepsize $\eta$ satisfies the condition $0<\eta\leq 1/2L$, then for all $N\geq 1$, we have *

[TABLE]

where $f^{\star}=f({\mathbf{x}}^{*},{\mathbf{y}}^{*})$ and $D=\|{\mathbf{x}}_{0}-{\mathbf{x}}^{*}\|^{2}+\|{\mathbf{y}}_{0}-{\mathbf{y}}^{*}\|^{2}$.

Proof.

From Lemma 2(a), we have that the iterates generated by the OGDA method satisfy Equation (2). On taking the sum of this relation from $k=0,\cdots,N-1$, we obtain for any ${\mathbf{z}}$

[TABLE]

Note that for any ${\mathbf{z}}_{1},{\mathbf{z}}_{2}\in\mathcal{D}$, we have:

[TABLE]

where we have used the fact that $\|{\mathbf{z}}-{\mathbf{z}}^{*}\|^{2}\leq 2\|{\mathbf{z}}_{0}-{\mathbf{z}}^{*}\|^{2}\ \text{for all}\ {\mathbf{z}}\in\mathcal{D}$ along with the fact that $\text{for all}\ {\mathbf{a}},{\mathbf{b}}\in\mathbb{R}^{d},\|{\mathbf{a}}+{\mathbf{b}}\|^{2}\leq 2\|{\mathbf{a}}\|^{2}+2\|{\mathbf{b}}\|^{2}$. As ${\mathbf{z}}_{-1}={\mathbf{z}}_{0}$ and $\eta\leq 1/2L$, for any ${\mathbf{z}}\in{\mathcal{D}}$ we have

[TABLE]

This inequality follows since:

[TABLE]

and for any ${\mathbf{x}},{\mathbf{y}}\in\mathcal{D}$, we have:

[TABLE]

Therefore, we have:

[TABLE]

which immediately gives us Inequality (40). Combining relation (40) with Proposition 1 we have that for all ${\mathbf{x}},{\mathbf{y}}\in\mathcal{D}$

[TABLE]

which gives us the following convergence rate estimate:

[TABLE]

where $f^{\star}=f({\mathbf{x}}^{*},{\mathbf{y}}^{*})$. ∎

Note that convergence in Theorem 2 is shown in terms of the Primal-Dual gap $\max_{{\mathbf{y}}:(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})\in\mathcal{D}}f(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})-\min_{{\mathbf{x}}:({\mathbf{x}},\hat{{\mathbf{y}}}_{N})\in\mathcal{D}}f({\mathbf{x}},\hat{{\mathbf{y}}}_{N})$ which is a common measure to capture closeness to the solution in convex-concave setting (see [Nemirovski, 2004]). Indeed, the duality gap is zero if and only if $(\hat{{\mathbf{x}}}_{N},\hat{{\mathbf{y}}}_{N})$ is a saddle point of the problem. The primal-dual gap also has the following game theoretic interpretation. If Player ${\mathbf{x}}$ is playing $\hat{{\mathbf{x}}}_{N}$, then $\max_{{\mathbf{y}}:(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})\in\mathcal{D}}f(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})$ quantifies how much Player ${\mathbf{y}}$ can gain by playing an action in the set $\mathcal{D}$. Similarly, if Player ${\mathbf{y}}$ is playing $\hat{{\mathbf{y}}}_{N}$, then $-\min_{{\mathbf{x}}:({\mathbf{x}},\hat{{\mathbf{y}}}_{N})\in\mathcal{D}}f({\mathbf{x}},\hat{{\mathbf{y}}}_{N})$ quantifies how much Player ${\mathbf{x}}$ can gain by playing an action in $\mathcal{D}$. Therefore, the quantity $\max_{{\mathbf{y}}:(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})\in\mathcal{D}}f(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})-\min_{{\mathbf{x}}:({\mathbf{x}},\hat{{\mathbf{y}}}_{N})\in\mathcal{D}}f({\mathbf{x}},\hat{{\mathbf{y}}}_{N})$ is a measure of the sum of how much each player can gain if they unilaterally deviate from the strategy $(\hat{{\mathbf{x}}}_{N},\hat{{\mathbf{y}}}_{N})$. This goes to zero at the Nash Equilibrium (saddle point), where no player can gain by unilaterally deviating from the equilibrium strategy.

Also, note that the result in Theorem 2 also implies that $|f({\hat{\mathbf{x}}}_{N},{\hat{\mathbf{y}}}_{N})-f^{*}|\leq 9LD/N$ as we show in the following corollary.

Corollary 1**.**

Suppose Assumptions 1, 2 and 3 hold. Let $\{{\mathbf{x}}_{k},{\mathbf{y}}_{k}\}$ be the iterates generated by the OGDA updates in (4). Consider the definition of the averaged iterates ${\hat{\mathbf{x}}}_{N},{\hat{\mathbf{y}}}_{N}$ in (3). If the stepsize $\eta$ satisfies the condition $0<\eta\leq 1/2L$, then for all $N\geq 1$, we have

[TABLE]

where $f^{\star}=f({\mathbf{x}}^{*},{\mathbf{y}}^{*})$.

Proof.

Note that $[\max_{{\mathbf{y}}:(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})\in\mathcal{D}}f(\hat{{\mathbf{x}}}_{N},{\mathbf{y}})-f^{\star}]$ and $[f^{\star}-\min_{{\mathbf{x}}:({\mathbf{x}},\hat{{\mathbf{y}}}_{N})\in\mathcal{D}}f({\mathbf{x}},\hat{{\mathbf{y}}}_{N})]$ are both nonnegative. To verify note that

[TABLE]

and

[TABLE]

(since $({\mathbf{x}}^{*},{\mathbf{y}}^{*})\in\mathcal{D}$). Further, note that $({\hat{\mathbf{x}}}_{N},{\hat{\mathbf{y}}}_{N})$ belongs to the set ${\mathcal{D}}$. Hence, it yields

[TABLE]

Also, we can show that

[TABLE]

Therefore, $|f({\hat{\mathbf{x}}}_{N},{\hat{\mathbf{y}}}_{N})-f^{\star}|\leq\frac{D(8L+\frac{1}{2\eta})}{N}$. ∎

The result in Corollary 1 shows that the function value of the averaged iterates generated by OGDA converges to the function value at a saddle point of problem (1) at a sublinear rate of $\mathcal{O}(1/k)$ when the function is smooth and convex-concave. To the best of our knowledge, this is the first non-asymptotic complexity bound for OGDA for the convex-concave setting. Moreover, note that without computing any extra gradient evaluation, i.e., computing only one gradient per iteration with respect to ${\mathbf{x}}$ and ${\mathbf{y}}$, OGDA recovers the convergence rate of proximal point method.

5 Extragradient Method

In this section, we consider finding a saddle point of a general smooth convex-concave function using the Extra-gradient (EG) method. Similar to our analysis of the OGDA method, we show that by interpreting the EG method as an approximation of the proximal point method it is possible to establish a convergence rate of $\mathcal{O}(1/k)$ through a simple analysis.

Consider the update of EG in which we first compute a set of mid-point iterates $\{{\mathbf{x}}_{k+\frac{1}{2}},{\mathbf{y}}_{k+\frac{1}{2}}\}$ using the gradients with respect to ${\mathbf{x}}$ and ${\mathbf{y}}$ at the current iterate

[TABLE]

Then, we compute the next iterates of the EG method $\{{\mathbf{x}}_{k+1},{\mathbf{y}}_{k+1}\}$ using the gradients at the mid-points $\{{\mathbf{x}}_{k+\frac{1}{2}},{\mathbf{y}}_{k+\frac{1}{2}}\}$, i.e.,

[TABLE]

We aim to show that EG, similar to OGDA, can be analyzed for convex-concave problems by considering it as an approximation of the proximal point. To do so, let us use the notation ${\mathbf{z}}=[{\mathbf{x}};{\mathbf{y}}]\in{\mathbb{R}}^{n+m}$ and $F({\mathbf{z}})=[\nabla_{{\mathbf{x}}}f({\mathbf{x}},{\mathbf{y}});-\nabla_{{\mathbf{y}}}f({\mathbf{x}},{\mathbf{y}})]\in{\mathbb{R}}^{n+m}$ to write the update of EG as

[TABLE]

To better highlight the connection between proximal point and EG, let us focus on the expression for the update of the mid-point iterates in EG. Considering the updates in (5), we have

[TABLE]

where the second equality follows by replacing ${\mathbf{z}}_{k}$ by its update ${\mathbf{z}}_{k-1}-\eta F({\mathbf{z}}_{k-\frac{1}{2}})$, and the second equality follows by considering the update ${\mathbf{z}}_{k-\frac{1}{2}}={\mathbf{z}}_{k-1}-\eta F({\mathbf{z}}_{k-1})$. Therefore, rearranging this equation, we can rewrite the updates as

[TABLE]

One can consider the expression $F({\mathbf{z}}_{k})-F({\mathbf{z}}_{k-1})$ as an approximation of the variation $F({\mathbf{z}}_{k+\frac{1}{2}})-F({\mathbf{z}}_{k-\frac{1}{2}})$. To be more precise, if we assume that the variations $F({\mathbf{z}}_{k})-F({\mathbf{z}}_{k-1})$ and $F({\mathbf{z}}_{k+\frac{1}{2}})-F({\mathbf{z}}_{k-\frac{1}{2}})$ are close to each other, i.e., $F({\mathbf{z}}_{k+\frac{1}{2}})-F({\mathbf{z}}_{k-\frac{1}{2}})\approx F({\mathbf{z}}_{k})-F({\mathbf{z}}_{k-1})$, then the update in (48) behaves like the proximal point update with. respect to the mid-point iterates, i.e.,

[TABLE]

We first derive a result similar to Proposition 2 for the specific case of EG iterates (Lemma 3(a)). We then show the the boundedness of the EG iterates in Lemma 3(b) (Note that the boundedness of the EG updates can also be deduced from the convergence results of Korpelevich [1976] and Monteiro & Svaiter [2010]).

Lemma 3**.**

*Let $\{{\mathbf{z}}_{k}\},\{{\mathbf{z}}_{k+\frac{1}{2}}\}$ be the iterates generated by the extra-gradient (EG) method introduced in (5). If Assumptions 1, 2 and 3 hold and the stepsize $\eta$ satisfies the condition $0<\eta<{1}/{L}$, then:

(a) The iterates $\{{\mathbf{z}}_{k}\},\{{\mathbf{z}}_{k+\frac{1}{2}}\}$ satisfy the following relation:*

[TABLE]

(b) The iterates $\{{\mathbf{z}}_{k}\},\{{\mathbf{z}}_{k+\frac{1}{2}}\}$ stay within the compact set ${\mathcal{D}}$ defined as

[TABLE]

where $({\mathbf{x}}^{*},{\mathbf{y}}^{*})={\mathbf{z}}^{*}\in\mathcal{Z}^{*}$ is a saddle point of the problem defined in (1). Moreover, the sum $\sum_{k=0}^{\infty}\|{\mathbf{z}}_{k+\frac{1}{2}}-{\mathbf{z}}_{k}\|^{2}$ is bounded above by

[TABLE]

Proof.

See Appendix B for the proof. ∎

The result in Lemma 3 shows that the iterates generated by the update of EG belong to a bounded and closed set. Now we use this result to show that the function value of the averaged iterates converges at a sublinear rate of $\mathcal{O}(1/k)$ to the function value at a saddle point for the EG method in the following theorem.

Theorem 3**.**

*Suppose Assumptions 1, 2 and 3 hold. Let $\{{\mathbf{x}}_{k+1/2},{\mathbf{y}}_{k+1/2}\}$ be the iterates generated by the EG updates in (5)-(5). Let the initial conditions satisfy ${\mathbf{x}}_{0}={\mathbf{x}}_{-1/2}$ and ${\mathbf{y}}_{0}={\mathbf{y}}_{-1/2}$ Consider the definition of the averaged iterates ${\hat{\mathbf{x}}}_{N},{\hat{\mathbf{y}}}_{N}$ in (3) and the compact convex set ${\mathcal{D}}$ in (51). If the stepsize $\eta$ satisfies the condition $\eta=\frac{\sigma}{L}$ for any $\sigma\in(0,1)$, then for all $N\geq 1$, we have *

[TABLE]

where $f^{\star}=f({\mathbf{x}}^{*},{\mathbf{y}}^{*})$ and $D=\|{\mathbf{x}}_{0}-{\mathbf{x}}^{*}\|^{2}+\|{\mathbf{y}}_{0}-{\mathbf{y}}^{*}\|^{2}$.

Proof.

See Appendix C for the proof. ∎

Now, similar to Corollary 1, we have:

Corollary 2**.**

Suppose Assumptions 1, 2 and 3 hold. Let $\{{\mathbf{x}}_{k+1/2},{\mathbf{y}}_{k+1/2}\}$ be the iterates generated by the EG updates in (5)-(5). Let the initial conditions satisfy ${\mathbf{x}}_{0}={\mathbf{x}}_{-1/2}$ and ${\mathbf{y}}_{0}={\mathbf{y}}_{-1/2}$ Consider the definition of the averaged iterates ${\hat{\mathbf{x}}}_{N},{\hat{\mathbf{y}}}_{N}$ in (3). If the stepsize $\eta$ satisfies the condition $\eta=\frac{\sigma}{L}$ for any $\sigma\in(0,1)$, then for all $N\geq 1$, we have

[TABLE]

where $f^{\star}=f({\mathbf{x}}^{*},{\mathbf{y}}^{*})$.

6 Discussion and Numerical Experiments

The main message of this work is that the OGDA algorithm obtains the same convergence rate of $\mathcal{O}(1/k)$, the best achievable rate (see [Nemirovski, 2004]), also achieved by EG. However, the advantage of OGDA is that we need only one gradient computation at each step, as opposed to two gradient computations needed in EG. This shows the computational advantage that OGDA has over EG.

We compare the performance of OGDA and EG in terms of gradient computations, on the bilinear minimax games considered in [Nemirovski, 2004], without any constraint. In particular, we consider the following minimax problem:

[TABLE]

where ${\mathbf{B}}\in\mathbb{R}^{n\times n}$ is a sparse random matrix generated as follows. Each element is nonzero independently with probability $p$. If an element is chosen to be non-zero, it is chosen uniformly from $[-1,1]$. We compare the number of gradient computations required to reach a desired accuracy level for this problem in Figure 1. As we observe, both EG and OGDA converge to the saddle point of the bilinear problem at a sublinear rate of $\mathcal{O}(1/k)$, but OGDA slightly outperforms EG in terms of number of gradient evaluations. Once again, this is due to the fact that for both descent and ascent updates of OGDA requires only one gradient computation each, while EG requires two gradient computations for both updates at each iteration.

Note that the Lipschitz constants for the considered problem can be estimated from data using standard line search techniques. In particular, Beck & Teboulle [2009] discuss a backward tracking algorithm (ISTA with backtracking) which can be used to estimate the Lipschitz constants, in particular $L_{xx}$ and $L_{yy}$. Several variants of this algorithm, including the Lipschitz line-search algorithm (Algorithm 2) in [Schmidt et al., 2015], can also be used to estimate the Lipschitz constants $L_{xx}$ and $L_{yy}$. For the specific case of saddle point problems, a recent paper [Hamedani & Aybat, 2018] proposes a line search algorithm, to estimate the Lipschitz constant $L_{xx},L_{xy},L_{yx}$ and $L_{yy}$. They propose an algorithm - Accelerated Primal Dual with backtracking (Algorithm 2.3) which uses a backtracking procedure, similar to [Malitsky & Pock, 2018], to locally estimate the Lipschitz constants of the problem. Also, regarding the initial error, we would like to highlight that in the analysis of convex minimization problems or convex-concave saddle point problems, we often have a term of the form $\|{\mathbf{x}}_{0}-{\mathbf{x}}^{*}\|^{2}$ in the upper bound (for instance see [Nesterov, 2013; Monteiro & Svaiter, 2010]) which shows the effect of initial error. This parameter is hard to estimate in general but can be upper bounded in specific cases. For example, if we are looking at for mixed strategies in zero-sum games, we know that we are looking for a solution lies in the probability simplex, so we can bound the initial error simply by the diameter of the simplex. In general, if we know that our iterates of the algorithm are going to lie in some compact set, we can upper bound the initial distance to the solution simply by the diameter of the compact set.

7 Conclusions

In this paper, we established convergence guarantees of the optimistic gradient ascent-descent (OGDA) and Extra-gradient (EG) methods for unconstrained, smooth, and convex-concave saddle point problems. In particular, we showed a sublinear convergence rate of $\mathcal{O}(1/k)$ in terms of function value error for both OGDA and EG by interpreting them as approximate variants of the proximal point method. This result leads to the first theoretical guarantee for OGDA in convex-concave saddle point problems. Moreover, it provides a simple and short proof for the convergence rate of EG in convex-concave saddle point problems when we measure optimality gap in terms of function value.

Appendix A Proof of Theorem 1

The update of the proximal point method can be written as:

[TABLE]

According to this update we can show that

[TABLE]

Now add and subtract the inner product $2\eta{\mathbf{z}}_{k+1}^{\top}F({\mathbf{z}}_{k+1})$ to the right hand side and regroup the terms to obtain

[TABLE]

Replace $F({\mathbf{z}}_{k+1})$ with $(1/\eta)(-{\mathbf{z}}_{k+1}+{\mathbf{z}}_{k})$ to obtain

[TABLE]

On rearranging the terms, we get the following

[TABLE]

Now, on substituting ${\mathbf{z}}={\mathbf{z}}^{*}$, and noting that $F({\mathbf{z}}_{k+1})^{\top}({\mathbf{z}}_{k+1}-{\mathbf{z}}^{*})\geq 0$, we have:

[TABLE]

and the proof of boundedness is complete.

On adding Equation (59) from $k=0,\cdots N-1$ and diving by $N$, we get:

[TABLE]

Now, using Proposition 1 we can write

[TABLE]

and the proof is complete.

Appendix B Proof of Lemma 3

(a) Considering the updates in (48) and (49) we can write the update of mid-points in EG as

[TABLE]

where,

[TABLE]

Therefore, we can simplify the last term in Equation (2) of Proposition 2 as follows:

[TABLE]

Using Lipschitz continuity of the operator $F$ (Lemma 1(b)) and Young’s inequality, we have

[TABLE]

Substituting the upper bound in (B) into Equation (2) of Proposition 2, implies that

[TABLE]

Since $\eta<1/L$, we have $-\frac{1}{2\eta}\|{\mathbf{z}}_{k+\frac{1}{2}}-{\mathbf{z}}_{k-\frac{1}{2}}\|^{2}+\frac{L}{2}\|{\mathbf{z}}_{k+\frac{1}{2}}-{\mathbf{z}}_{k-\frac{1}{2}}\|^{2}\leq 0$ and therefore

[TABLE]

which completes the proof of Part (a).

(b) Based on the update of EG in (5), we can write

[TABLE]

Now we proceed to bound the difference $\|{\mathbf{z}}_{k+\frac{1}{2}}-{\mathbf{z}}_{k+1}\|^{2}$. Using the fact that the operator $F$ is $L$-Lipschitz (Lemma 1(b)), we have

[TABLE]

Substituting this upper bound back into (69) and taking ${\mathbf{z}}={\mathbf{z}}^{*}$ implies

[TABLE]

Further, since the operator $F$ is monotone, we have

[TABLE]

where in the first inequality we used the fact that $F({\mathbf{z}}^{*})={\mathbf{0}}$ (Lemma 1(c)), and the last inequality holds due to monotonicity of $F$ (Lemma 1(a)). Therefore, we can replace the inner product $2({\mathbf{z}}^{*}-{\mathbf{z}}_{k+\frac{1}{2}})^{\top}({\mathbf{z}}_{k+1}-{\mathbf{z}}_{k})$ in (B) by its lower bound [math] to obtain

[TABLE]

The result in (73) shows that the seqeunce $\|{\mathbf{z}}_{k}-{\mathbf{z}}^{*}\|^{2}$ is non-increasing. Therefore, for any iterate $k$, it holds that

[TABLE]

Now, for all $k\geq 0$, we have:

[TABLE]

where the first inequality follows from the fact that $\text{for all}\ {\mathbf{a}},{\mathbf{b}}\in\mathbb{R}^{d},\|{\mathbf{a}}+{\mathbf{b}}\|^{2}\leq 2\|{\mathbf{a}}\|^{2}+2\|{\mathbf{b}}\|^{2}$, the second inequality follows from (73) and the third inequality follows from (74). Therefore from (74) and (B), since $0<1-\eta^{2}L^{2}<1$, we see that the iterates $\{{\mathbf{z}}_{k}\},\{{\mathbf{z}}_{k+\frac{1}{2}}\}$ belong to the compact set ${\mathcal{D}}$ defined in (51).

Now by summing both sides of (73) for $k=0,\dots,\infty$, we obtain

[TABLE]

Therefore, by regrouping the terms we obtain

[TABLE]

and the claim in (52) follows.

Appendix C Proof of Theorem 3

Using Equation (3) of Lemma 3(a), summing it from $k=0,\cdots,N-1$ and dividing by $N$, we obtain

[TABLE]

The bound in Equation (52) from Lemma 3(b) yields

[TABLE]

where in the last inequality we use Lipschitz continuity of the operator $F$ (Lemma 1(b)) and the fact that $\eta=\frac{\sigma}{L}$. Note that for any ${\mathbf{z}}_{1},{\mathbf{z}}_{2}\in\mathcal{D}$, we have:

[TABLE]

Therefore, for any point ${\mathbf{z}}$ in the set ${\mathcal{D}}$, we can substitute the preceding relation in Equation (79) to get

[TABLE]

Now, using Proposition 1 we have that for all ${\mathbf{x}},{\mathbf{y}}\in\mathcal{D}$:

[TABLE]

where $\hat{{\mathbf{x}}}_{N}=\frac{1}{N}\sum_{k=0}^{N-1}{\mathbf{x}}_{k+1/2}$ and $\hat{{\mathbf{y}}}_{N}=\frac{1}{N}\sum_{k=0}^{N-1}{\mathbf{y}}_{k+1/2}$ which gives us the following convergence result:

[TABLE]

where $f^{\star}=f({\mathbf{x}}^{*},{\mathbf{y}}^{*})$.