An Optimistic Gradient Tracking Method for Distributed Minimax Optimization

TL;DR

This paper introduces a distributed optimistic gradient tracking method, DOGT, for minimax optimization over networks, achieving linear convergence and optimal rates with robustness to heterogeneity.

Contribution

The paper proposes DOGT and ADOGT algorithms that improve convergence rates for distributed minimax problems, incorporating optimism and acceleration techniques.

Findings

DOGT achieves linear convergence for strongly convex-strongly concave problems.

ADOGT attains optimal convergence rate and communication complexity.

Numerical experiments confirm the effectiveness of the proposed methods.

Abstract

This paper studies the distributed minimax optimization problem over networks. To enhance convergence performance, we propose a distributed optimistic gradient tracking method, termed DOGT, which solves a surrogate function that captures the similarity between local objective functions to approximate a centralized optimistic approach locally. Leveraging a Lyapunov-based analysis, we prove that DOGT achieves linear convergence to the optimal solution for strongly convex-strongly concave objective functions while remaining robust to the heterogeneity among them. Moreover, by integrating an accelerated consensus protocol, the accelerated DOGT (ADOGT) algorithm achieves an optimal convergence rate of $\mathcal{O} \left( \kappa \log \left( \epsilon ^{-1} \right) \right)$ and communication complexity of $\mathcal{O} \left( \kappa \log \left( \epsilon ^{-1} \right) /\sqrt{1-\sqrt{\rho _W}} \right)$ for a suboptimality level of $\epsilon>0$, where $\kappa$ is the condition number of the objective function and $\rho_W$ is the spectrum gap of the network. Numerical experiments illustrate the effectiveness of the proposed algorithms.

Equations

$$\underset{x\in {\mathbb{R}}^p}{\min }\underset{y\in {\mathbb{R}}^d}{\max }f\left(x,y\right):=\frac{1}{n}\sum _{i=1}^n{f}_i\left(x,y\right),$$

$$\underset{\mathbf{x}\in {\mathbb{R}}^{n\times p}}{\min }\underset{\mathbf{y}\in {\mathbb{R}}^{n\times d}}{\max }F\left(\mathbf{x},\mathbf{y}\right)=\frac{1}{n}\sum _{i=1}^n{f}_i\left(x_i,y_i\right),$$

$$\text{s.t.}{x}_i=x_j,y_i=y_j,\forall i,j=1,\dots ,n$$

$$\mathbf{x}$$

$$\mathbf{y}$$

$$f_i\left(x^{\prime },y\right)-f_i\left(x,y\right)$$

$$⩾⟨{\nabla }_x{f}_i\left(x,y\right),x^{\prime }-x⟩+\frac{\mu }{2}{\Vert x-x^{\prime }\Vert }^2,$$

$$f_i\left(x,y^{\prime }\right)-f_i\left(x,y\right)$$

$$⩽⟨{\nabla }_y{f}_i\left(x,y\right),y^{\prime }-y⟩-\frac{\mu }{2}{\Vert y-y^{\prime }\Vert }^2.$$

$${\Vert {\nabla }_z{f}_i\left(x,y\right)-{\nabla }_z{f}_i\left(x^{\prime },y^{\prime }\right)\Vert }^2$$

$$⩽L^2\left({\Vert x-x^{\prime }\Vert }^2+{\Vert y-y^{\prime }\Vert }^2\right).$$

$$f_i\left(x_i,y_i\right)$$

$$+{\underset{\mathrm{similarity}\mathrm{in}x}{\underbrace{\left(p_{i,k}-{\nabla }_x{f}_i\left(x_{i,k},y_{i,k}\right)\right)}}}^{\top }\left(x_i-x_{i,k}\right)$$

$$+{\underset{\mathrm{similarity}\mathrm{in}y}{\underbrace{\left(q_{i,k}-{\nabla }_y{f}_i\left(x_{i,k},y_{i,k}\right)\right)}}}^{\top }\left(y_i-y_{i,k}\right),$$

$$x_{i,k+1}=x_{i,k}$$

$$-\gamma \left(p_{i,k}-{\nabla }_x{f}_i\left(x_{i,k},y_{i,k}\right)\right).$$

$${\nabla }_x{f}_i\left(x_{i,k+1},y_{i,k+1}\right)$$

$$\approx 2{\nabla }_x{f}_i\left(x_{i,k},y_{i,k}\right)-{\nabla }_x{f}_i\left(x_{i,k-1},y_{i,k-1}\right),$$

$$x_{i,k+1}$$

$$y_{i,k+1}$$

$$p_{i,k+1}$$

$$q_{i,k+1}$$

$${\theta }_{i,k+1}\leftarrow \sum _{j\in {\mathcal{N}}_i}{w}_{i,j}{\theta }_{i,k+1}.$$

$${\nabla }_x{F}_k$$

$${\nabla }_y{F}_k$$

$${\mathbf{x}}_{k+1}$$

$${\mathbf{y}}_{k+1}$$

$${\mathbf{p}}_{k+1}$$

$${\mathbf{q}}_{k+1}$$

$${\mathbf{p}}_k$$

$${\mathbf{q}}_k$$

$${\bar{x}}_{k+1}$$

$${\bar{y}}_{k+1}$$

$${\bar{p}}_{k+1}$$

$${\bar{q}}_{k+1}$$

$${\theta }_{i,k+1}\leftarrow \left(1+\eta \right)\sum _{j\in {\mathcal{N}}_i}{w}_{i,j}{\theta }_{j,k+1}-\eta {\theta }_{i,k+1},$$

$$M_{t+1}=\left(1+\eta \right)W{M}_t-\eta M_{t-1},$$

$${\mathbf{z}}_k=\left[{\mathbf{x}}_k,{\mathbf{y}}_k\right],{\mathbf{r}}_k=\left[{\mathbf{p}}_k,-{\mathbf{q}}_k\right],\nabla F_k=\left[{\nabla }_x{F}_k,-{\nabla }_y{F}_k\right].$$

$${\bar{z}}_{k+1}={\bar{z}}_k-\gamma \frac{1^{\top }}{n}\nabla F_{k+1}+\gamma \frac{1^{\top }}{n}{\mathbf{\epsilon }}_k,$$

$${\bar{z}}_{k+1}-\gamma \frac{1^{\top }}{n}\left(\nabla F_{k+1}-\nabla F_k\right)$$

Full text

I Introduction

Minimax optimization has gained significant attention over the past decade due to its broad applications in robust optimization [1, 2], game theory [3], and generative adversarial networks (GANs) [4, 5], among others. With the increasing scale of data and devices [6], distributed optimization has emerged as a key approach in large-scale optimization [7], machine learning [8], and control [9].

In this paper, we consider the distributed minimax optimization problem jointly solved by a network of $n$ nodes:

[TABLE]

where $f_{i}$, $i=1,\dots,n$, is the local objective function, $x\in\mathbb{R}^{p}$ and $y\in\mathbb{R}^{d}$ are primal and dual variables to be minimized and maximized, respectively. This formulation is general, as it recovers the traditional distributed minimization problem [7] as a special case, thereby capturing a broader class of applications. For instance, in distributed training of Wasserstein GANs [10, 11], nodes collaborate to minimize the generator while simultaneously maximizing the discriminator, thus enabling data-parallel accelerated training. A straightforward approach to solving this problem is to apply centralized minimax algorithms, such as gradient descent ascent (GDA) [12], optimistic gradient descent ascent (OGDA) [13] and extragradient (EG) [14], in combination with the averaging consensus protocol [15]. For instance, Deng and Mahdavi [16] proposed Local SGDA, which integrates FedAvg from federated learning [17] with stochastic GDA to reduce communication costs. Chen et al. [18] introduced a variance-reduction-based decentralized GDA method, achieving improved sample complexity under the stronger assumption of average smoothness for each sample. However, as shown in [19], centralized GDA methods do not achieve linear convergence for bilinear functions. We will show later that this limitation also applies to the distributed GDA (DGDA) algorithm.

OGDA and EG methods have gained popularity in recent literature due to their ability to achieve linear convergence rates for strongly convex-strongly concave and bilinear objective functions [19, 20], as well as their effectiveness in training GANs [13, 21]. In decentralized settings, Liu et al. [10] proposed a distributed optimistic gradient descent ascent method, DPOSG, for training large-scale GANs and analyzed its convergence for nonconvex-nonconcave (NC-NC) objective functions. Beznosikov et al. [22] introduced an EG-based FedAvg method and proved that with an accelerated consensus protocol, it achieves a near-optimal convergence rate, matching the lower bound for (strongly) convex-(strongly) concave objective functions up to logarithmic factors. Liu et al. [23] developed a decentralized proximal point method for NC-NC problems, which requires solving a subproblem at each iteration. However, the above methods establish convergence under assumptions such as uniformly bounded gradient norms [10] or bounded feasible domains [23, 22]. Otherwise, a decaying stepsize is required to remove a steady-state error [24]. These assumptions restrict the applicability of their theoretical results in many real-world tasks due to their inability to handle the heterogeneity of local objective functions [8], i.e., the gradients of each $f_{i}$ differ, leading to distinct local optimas. This heterogeneity is recognized as a fundamental challenge in distributed optimization.

Gradient tracking (GT) methods [25, 26] are widely used in distributed minimization to mitigate the effects of data/function heterogeneity. Recent studies have explored their application to minimax optimization. For instance, Wai et al. [27] introduced a GDA-based GT method for multi-agent reinforcement learning and established a linear convergence rate for SC-SC settings. Mukherjee et al. [28] combined GT with EG methods, achieving linear convergence for strongly convex-strongly concave (SC-SC) objectives without assuming a bounded gradient norm or feasible domain. However, there remains a gap between the convergence rates in [27, 28] and the established lower bound in [22]. Another line of research utilizes second-order similarity based on the mirror-descent method [29, 30, 31], where the difference between the second-order derivative matrices of $f_{i}$ is bounded by a finite quantity $\delta$. This approach improves algorithmic efficiency when $\delta$ is smaller than the smoothness condition number. However, these methods often incur high computational costs due to the need to solve a subproblem.

To address these challenges, inspired by the mirror-descent-type method in decentralized settings [29, 30], we propose a distributed optimistic gradient tracking method, termed DOGT, for solving Problem (1). This algorithm solves a surrogate objective function that captures the similarity between the local gradient and an estimator of the global gradient, using one-step approximate solutions to avoid the need for inner loops to solve subproblems. Theoretically, we prove that DOGT achieves a linear convergence rate and outperforms the result in [28] by a factor of $\mathcal{O}\left(\kappa^{1/3}\right)$ when the graph connectivity is strong. Furthermore, by integrating an accelerated consensus protocol with prior knowledge of the graph, we propose ADOGT and prove that it achieves an optimal convergence rate of $\mathcal{O}\left(\kappa\log\left(\epsilon^{-1}\right)\right)$ and communication complexity of $\mathcal{O}\left(\kappa\log\left(\epsilon^{-1}\right)/\sqrt{1-\sqrt{\rho_{W}}}\right)$, matching the lower bound established in [22] for deterministic settings under the assumptions considered in this work. Numerical results validate our theoretical findings.

Paper organization. The rest of the paper is structured as follows: Section II formulates the distributed minimax optimization problem with several common assumptions and presents the algorithm design of DOGT and ADOGT. Section III provides the main convergence results of the proposed algorithms. Section IV presents numerical experiments to validate our theoretical findings. Finally, Section V concludes the paper, and several supporting lemmas for the proof of the main results are provided in the appendix.

Notations. Throughout this paper, we adopt the following notations: $\left<\cdot,\cdot\right>$ is the inner product of vectors, $\left\|\cdot\right\|$ represents the Frobenius norm, $\lceil\cdot\rceil$ indicates the ceiling operation, $\mathbf{1}$ represents the all-ones vector, $\mathbf{I}$ denotes the identity matrix, and $\mathbf{J}=\mathbf{1}\mathbf{1}^{\top}/n$ denotes the averaging matrix.

II Problem Formulation and Algorithm Design

II-A Distributed Minimax Optimization Problem

To solve the distributed minimax optimization problem (1) in a decentralized manner, we consider each node $i\in[n]$ maintaining a local copy of the global primal and dual variables, i.e., $x_{i}\in\mathbb{R}^{p},y_{i}\in\mathbb{R}^{d}$, and optimize the following constrained problem, which has the same optimal solution as the original problem (1):

[TABLE]

where

[TABLE]

are the collections of the primal and dual variables, respectively, which are required to reach consensus. Moreover, the nodes are connected over a decentralized network, and its topology is modeled as an undirected graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, where $\mathcal{V}=\{1,2,...,n\}$ denotes the set of agents, and $\mathcal{E}\subseteq\mathcal{V}\times\mathcal{V}$ denotes the set of edges consisting of ordered pairs $(i,j)$ modeling the communication link from $j$ to $i$. Each node $i$ only communicates with its neighbor $\mathcal{N}_{i}=\left\{j\,|\,j\neq i,\,\left(i,j\right)\in\mathcal{E}\right\}$, including $i$ itself. Then, we make the following commonly used assumptions on the differentiable objective function $f_{i}$, its gradient $\nabla f_{i}$, and the graph $\mathcal{G}$.

Assumption 1** (Convexity and Concavity)**

Each objective function $f_{i}\left(x,y\right)$ is $\mu$-strongly convex in $x$ and $\mu$-strongly concave in $y$, i.e., $\forall x,x^{\prime}\in\mathbb{R}^{p}$, $\forall y,y^{\prime}\in\mathbb{R}^{d}$ and $\mu>0$,

[TABLE]

Assumption 2** (Smoothness)**

Each objective function $f_{i}\left(x,y\right)$ is jointly $L$-smooth in $x$ and $y$, i.e., $\forall x,x^{\prime}\in\mathbb{R}^{p}$ and $\forall y,y^{\prime}\in\mathbb{R}^{d}$, there exists a constant $L>0$ such that for $z\in\left\{x,y\right\}$,

[TABLE]

Assumption 3** (Graph connectivity)**

The weight matrix $W=\left[w_{i,j}\right]_{i,j=1}^{n}$ induced by graph $\mathcal{G}$ is doubly stochastic, i.e., $W\mathbf{1}=\mathbf{1},\mathbf{1}^{\top}W=\mathbf{1}^{\top}$ and $\rho_{W}:=\left\|W-\mathbf{J}\right\|_{2}^{2}<1$.

II-B Algorithm Design

Motivated by the SONATA algorithm [29] originally designed for minimization problems, we propose a distributed optimistic gradient tracking (DOGT) algorithm for solving the distributed minimax problem (2). In particular, we consider the following surrogate function for minimizing on $x_{i}\in\mathbb{R}^{p}$ and maximizing on $y_{i}\in\mathbb{R}^{d}$:

[TABLE]

where $p_{i,k}$ and $q_{i,k}$ are the gradient tracking variables used for estimating the global gradient at iteration $k$ with respect to $x$ and $y$, receptively. This surrogate function captures the difference between the local and global gradients. Solving it in a decentralized manner helps to asymptotically reduce this gap and approximate a centralized optimistic method, thereby improving the convergence [29]. Specifically, according to the first-order optimality condition of (3), and taking the primal variable $x$ as an example, we derive the following proximal point method:

[TABLE]

To obtain an algorithm that is easy to deploy without requiring the exact solution of the surrogate function (3), we use the approximation

[TABLE]

which has an error bounded by $o\left(\gamma^{2}\right)$ [19]. Based on this, we propose the DOGT algorithm, an optimistic gradient descent ascent method with gradient tracking. The pseudo-code for DOGT is provided in Algorithm 1. Notably, DOGT adopts a single-loop structure, making it more suitable for practical deployment than mirror-descent-based methods [29, 30], particularly in scenarios with data heterogeneity.

For brevity, we introduce the following notations for the gradients of all nodes at iteration $k$:

[TABLE]

Then, the DOGT algorithm can be rewritten in the following compact form:

[TABLE]

where the collection of the gradient tracking variables for the primal and dual decision variables are denoted as follows:

[TABLE]

Furthermore, taking the average over all nodes on both sides of the algorithm yields the following equations:

[TABLE]

It can be observed that, on average, DOGT employs a centralized OGDA update scheme, with the gradient tracking variables asymptotically aligning with the average gradient across all nodes. The consensus of these variables is ensured by the weighting matrix $W$, which satisfies Assumption 3.

To further enhance the convergence of DOGT, we incorporate the accelerated gossip consensus protocol [32] and obtained ADOGT, leveraging prior knowledge of the spectrum gap $\rho_{W}$. Specifically, we replace the communication step of DOGT (cf. line 5 in Algorithm 1) with the following update, executed a finite number of $T$ times at each iteration:

[TABLE]

where $\eta=\left(1-\sqrt{1-\rho_{W}}\right)/\left(1+\sqrt{1-\rho_{W}}\right)$. Then, we obtain a generated weight matrix $M_{T}$ of ADOGT as follows:

[TABLE]

where $t=0,1,\dots,T-1$, and $M_{-1}=M_{0}=\mathbf{I}$. We will show that ADOGT achieves the optimal convergence rate with this accelerated weight matrix (cf., Theorem 2).

III Convergence Analysis

In this section, we provide the convergence analysis of DOGT and ADOGT for SC-SC smooth objective functions. For simplicity, we further denote

[TABLE]

Then, letting $\boldsymbol{\varepsilon}_{k}=\nabla F_{k+1}-\nabla F_{k}-\nabla F_{k}+\nabla F_{k-1}$, we get the update rule of $\bar{z}_{k}:=\mathbf{1}^{\top}\mathbf{z}_{k}/n$ as follows:

[TABLE]

and

[TABLE]

III-A Linear Convergence

To prove the convergence of DOGT, we define the following Lyapunov function:

[TABLE]

where $c_{1}$ and $c_{2}$ are coefficients to be designed, and the optimality gap to the optimal solution $z^{*}:=\left[x^{*},y^{*}\right]$ of problem (2) is defined as

[TABLE]

Then, with the help of Lemmas 1-4 in the appendix, the following theorem shows a linear convergence rate of DOGT.

Theorem 1

Consider the DOGT algorithm as depicted in (6). Suppose Assumptions 1-3 hold. Let the stepsize

[TABLE]

Then, we have for all $k\geqslant 0$,

[TABLE]

Proof:

By Lemma 1 and Lemma 4, we can obtain that

[TABLE]

Letting $\gamma\leqslant 1/(8L)$ such that

[TABLE]

we can obtain

[TABLE]

Then, combining Lemma 2 and 3 , we can obtain the contraction of the Lyapunov function:

[TABLE]

Set the parameters of the Lyapunov function as follows:

[TABLE]

Then, letting the stepsize further satisfy (14) such that the coefficients of the last four terms on the right-hand side of the inequality are all less than or equal to 0, we complete the proof. ∎

Remark 1

Theorem 1 shows that DOGT converges to the optimal solution of problem (2) at a linear rate. And, by the upper bound of the stepsize (14), we can drive that it reaches a suboptimality level of $\epsilon>0$ in at most $K$ iterations:

[TABLE]

where $\kappa:=L/\mu$ denotes the condition number of the overall objective function, $\mathcal{O}\left(\cdot\right)$ hides the constants. Compared to the GT-EG method [28], DOGT achieves at least the same convergence rate and improves it by a factor of $\mathcal{O}\left(\kappa^{1/3}\right)$ when $\rho_{W}$ is small, indicating strong network connectivity.

III-B Optimal Convergence Rate

For the ADOGT algorithm with accelerated consensus protocol (8), the following theorem gives the optimal convergence rate and communication complexity matching the existing lower bound in deterministic settings [22].

Theorem 2

Suppose Assumptions 1-3 hold. Let the stepsize satisfy (14), and the number of communication steps at each round $T=\lceil\ln\left(2\right)/\sqrt{1-\sqrt{\rho_{W}}}\rceil$. Then, for a given $\rho_{W}$, the ADOGT algorithm achieves a linear rate of $\mathcal{O}\left(\kappa\log\left(\epsilon^{-1}\right)\right)$, and the number of communication rounds $R$ required to reach a suboptimality level of $\epsilon>0$ is

[TABLE]

Proof:

The proof of ADOGT follows a similar approach to that of DOGT, with the weight matrix $W$ replaced by $M_{T}$. In specific, after executing $T$ steps of the accelerated consensus (8), we obtain an equivalent weight matrix $M_{T}$ generated by (9). Then, by Proposition 3 in [32], we get

[TABLE]

Letting the number of communication steps at each round $T=\lceil\ln(2)/\sqrt{1-\sqrt{\rho_{W}}}\rceil$, we have $1-\rho_{M}\geqslant 1/2$. Then, replacing $\rho_{W}$ in the iteration complexity of DOGT in (20) with $\rho_{M}$ and noticing that the total number of communication steps equals the iterations multiplied by $T$, we obtain the optimal linear convergence rate and communication complexity. ∎

IV Numerical Experiments

In this section, we conduct numerical experiments to compare the convergence performance between DGDA, D-OGDA, DOGT, and ADOGT. To this end, we consider the following synthetic example:

[TABLE]

where $i\in[n]$, $a_{i}\in\mathbb{R}^{p}$ and $b_{i}\in\mathbb{R}^{d}$ are different for each nodes. We consider $n=16$ nodes connected as an undirected ring graph. We set the dimensions $p=d=2$, the regularization constant $\mu=0.1$, and the stepsize $\gamma=0.1$, respectively. For ADOGT, we set the number of communication steps at each round $T=4$ as suggested by Theorem 2. Moreover, we set the optimal solution $z^{*}=\left[0,0;0,0\right]$ by choosing $\sum\nolimits_{i=1}^{n}{a_{i}}=\sum\nolimits_{i=1}^{n}{b_{i}}=[0;0]$.

In Fig. 1, we plot the trajectory with respect to the primal and dual decision variables, the residual $1/n\left\|\mathbf{z}_{k}-\mathbf{1}z^{*}\right\|^{2}$, and the consensus error $1/n\left\|\mathbf{z}_{k}-\mathbf{1}\bar{z}\right\|$ for each algorithm, respectively. The trajectories show that DOGT and ADOGT converge to the optimal solution, with ADOGT achieving faster convergence due to the integration of an accelerated consensus protocol. This demonstrates the robustness of GT-based methods against function heterogeneity. Instead, D-OGDA converges to a non-optimal point, and DGDA exhibits periodic oscillations without achieving convergence. Their non-zero residuals further support this behavior. Additionally, we observe that the consensus errors of DGDA and D-OGDA do not converge to zero. This is due to the heterogeneity of the objective functions, as also observed in [24], which leads to inconsistent optimal points across nodes. In contrast, DOGT and ADOGT remain robust to this issue. These results demonstrate the effectiveness of the proposed algorithms.

V Conclusion

In this paper, we have proposed a distributed optimistic gradient tracking method, DOGT, along with its accelerated variant, ADOGT, for solving distributed minimax optimization problems over networks. We have also provided rigorous theoretical analysis to show that DOGT achieves a linear convergence rate to the optimal solution for SC-SC objective functions. Furthermore, the proposed ADOGT integrated with accelerated consensus protocol achieves an optimal convergence rate and communication complexity that matches the existing lower bound. Numerical experiments demonstrate the effectiveness of the proposed algorithms. Future work will focus on the design and analysis of algorithms in stochastic and non-convex settings, as well as exploring their potential applications in the distributed training of GANs.

Appendix A Supporting Lemmas

In this section, we introduce several supporting lemmas for the proof of the main results, highlighting important contraction properties of the consensus error, gradient tracking error, and optimality gap.

Lemma 1

Suppose Assumptions 2 and 3 hold. Then, we have for all $k\geqslant 0$,

[TABLE]

Proof:

By the updates of DOGT (6) and (11), we get

[TABLE]

where we used the fact $\left\|W-\mathbf{I}\right\|^{2}\leqslant 2$ and the smoothness of $f_{i}$. Then, noticing that $\bar{r}_{k}=\frac{\mathbf{1}^{\top}}{n}\nabla F_{k+1}$, we can obtain the result in (24). ∎

The following lemma shows the contraction of the consensus error.

Lemma 2** (Consensus error)**

Suppose Assumptions 2 and 3 hold. Let the stepsize satisfy $\gamma\leqslant 1/\left(4L\right)$, we get $\forall k\geqslant 0$,

[TABLE]

Proof:

By the update rule of DOGT (6) and Young’s inequality, we get

[TABLE]

Then, letting $\lambda=\frac{1-\rho_{W}}{2\rho_{W}}$, and using the smoothness of $f_{i}$ and Assumption 3, we complete the proof. ∎

Next, we have the following lemma showing the contraction of the gradient tracking error.

Lemma 3** (Gradient tracking error)**

Suppose Assumptions 2 and 3 hold. Let the stepsize satisfy

[TABLE]

Then, we get for all $k\geqslant 0$,

[TABLE]

Proof:

According to the updates of DOGT (6), we have

[TABLE]

where we used Young’s inequality with $\lambda=\frac{1-\rho_{W}}{2\rho_{W}}$ and the smoothness of $f_{i}$. Then, letting the stepsize satisfy (26) and applying Lemma 1, we complete the proof. ∎

We are now in a position to show the key lemma on the contraction of the optimality gap defined in (13).

Lemma 4** (Optimality gap)**

Suppose Assumptions 1-3 hold. Let the stepsize satisfy

[TABLE]

Then, we have for all $k\geqslant 0$,

[TABLE]

Proof:

According to (11), we have

[TABLE]

Since $\varXi_{k+1}=\varXi_{k}-\gamma\frac{\mathbf{1}^{\top}}{n}\nabla F_{k+1}$, we have

[TABLE]

Noticing that $a^{2}-2ab=\left(a-b\right)^{2}-b^{2},\forall a,b\in\mathbb{R}$, we get

[TABLE]

For the inner product term, we have

[TABLE]

Then, by Assumptions 1 and 2, we have

[TABLE]

Putting it back to (30) and using $f\left(\bar{z}_{k+1}\right)-f\left(z^{*}\right)\geqslant\frac{\mu}{2}\left\|\bar{z}_{k+1}-z^{*}\right\|^{2}$, we get

[TABLE]

Noticing that

[TABLE]

and

[TABLE]

we get

[TABLE]

Then, according to the obtained inequalities (24) and (25) and letting the stepsize satisfy (28), we can obtain that

[TABLE]

Then, noticing that $\left\|\nabla f\left(\bar{z}_{k}\right)\right\|^{2}\leqslant\frac{1}{n}\left\|\nabla F\left(\mathbf{1}\bar{z}_{k}\right)\right\|^{2}$ and $-\left\|\nabla F_{k}\right\|^{2}\leqslant L^{2}\left\|\mathbf{z}_{k}-\mathbf{1}\bar{z}_{k}\right\|^{2}-\frac{1}{2}\left\|\nabla F\left(\mathbf{1}\bar{z}_{k}\right)\right\|^{2}$, we complete the proof. ∎