Graphon Particle Systems, Part II: Dynamics of Distributed Stochastic Continuum Optimization

TL;DR

This paper investigates distributed stochastic optimization over graphons with infinitely many nodes, proposing algorithms and proving convergence properties, including consensus and optimality, in the continuum limit.

Contribution

It introduces stochastic gradient descent and gradient tracking algorithms for graphon-based networks and provides convergence analysis with novel differential inequality techniques.

Findings

Nodes' states achieve consensus on the graphon.

States converge to the global minimizer under strong convexity.

Second moments of node states are uniformly bounded.

Abstract

We study the distributed optimization problem over a graphon with a continuum of nodes, which is regarded as the limit of the distributed networked optimization as the number of nodes goes to infinity. Each node has a private local cost function. The global cost function, which all nodes cooperatively minimize, is the integral of the local cost functions on the node set. We propose stochastic gradient descent and gradient tracking algorithms over the graphon. We establish a general lemma for the upper bound estimation related to a class of time-varying differential inequalities with negative linear terms, based upon which, we prove that for both kinds of algorithms, the second moments of the nodes' states are uniformly bounded. Especially, for the stochastic gradient tracking algorithm, we transform the convergence analysis into the asymptotic property of coupled nonlinear differential inequalities with time-varying coefficients and develop a decoupling method. For both kinds of algorithms, we show that by choosing the time-varying algorithm gains properly, all nodes' states achieve $\mathcal{L}^{\infty}$-consensus for a connected graphon. Furthermore, if the local cost functions are strongly convex, then all nodes' states converge to the minimizer of the global cost function and the auxiliary states in the stochastic gradient tracking algorithm converge to the gradient value of the global cost function at the minimizer uniformly in mean square.