Convergence Analysis of Continuous-Time Distributed Stochastic Gradient Algorithms

TL;DR

This paper introduces a continuous-time distributed stochastic gradient algorithm for multi-agent systems to collaboratively minimize convex functions, proving convergence in expectation under certain conditions.

Contribution

It develops a novel continuous-time framework for distributed stochastic optimization with time-varying directed graphs and proves asymptotic convergence using advanced mathematical tools.

Findings

Agents' states asymptotically reach a common minimizer in expectation

The proposed algorithm converges under mild connectivity and convexity assumptions

Simulation confirms the theoretical convergence results

Abstract

In this paper, we propose a new framework to study distributed optimization problems with stochastic gradients by employing a multi-agent system with continuous-time dynamics. Here the goal of the agents is to cooperatively minimize the sum of convex objective functions. When making decisions, each agent only has access to a stochastic gradient of its own objective function rather than the real gradient, and can exchange local state information with its immediate neighbors via a time-varying directed graph. Particularly, the stochasticity is depicted by the Brownian motion. To handle this problem, we propose a continuous-time distributed stochastic gradient algorithm based on the consensus algorithm and the gradient descent strategy. Under mild assumptions on the connectivity of the graph and objective functions, using convex analysis theory, the Lyapunov theory and Ito formula, we prove that the states of the agents asymptotically reach a common minimizer in expectation. Finally, a simulation example is worked out to demonstrate the effectiveness of our theoretical results.