A Poisson Jump-driven SDE Approach to Distributed Gradient Descent with Sparse Communication

TL;DR

This paper introduces a novel framework using Poisson jump-driven stochastic differential equations to model asynchronous, sparse communication in distributed gradient descent, providing theoretical guarantees for convergence and stability.

Contribution

It develops a new SDE-based approach to incorporate asynchronous, sparse communication into distributed optimization algorithms, with proven convergence bounds.

Findings

Establishes communication rate bounds for stability and convergence.

Demonstrates the approach's effectiveness through numerical simulation.

Provides theoretical analysis for unconstrained quadratic optimization.

Abstract

To bridge the gap between idealised communication models and the stochastic reality of networked systems, we introduce a framework for embedding asynchronous communication directly into algorithm dynamics using stochastic differential equations (SDE) driven by Poisson Jumps. We apply this communication-aware design to the continuous-time gradient flow, yielding a distributed algorithm where updates occur via sparse Poisson events. Our analysis establishes communication rate bounds for asymptotic stability and, crucially, a higher, yet sparse, rate that provably any desired exponential convergence performance slower than the nominal, centralized flow. These theoretical results, shown for unconstrained quadratic optimisation, are validated by a numerical simulation.