On Distributed Stochastic Gradient Algorithms for Global Optimization

TL;DR

This paper investigates distributed stochastic gradient algorithms for nonconvex optimization, demonstrating convergence to global minima under relaxed assumptions compared to previous methods.

Contribution

It introduces a convergence analysis that relaxes the bounded-gradient-dissimilarity condition in distributed nonconvex optimization.

Findings

Convergence to global minima achieved with relaxed assumptions.

Distributed algorithms effectively escape local minima.

Theoretical guarantees under less restrictive conditions.

Abstract

The paper considers the problem of network-based computation of global minima in smooth nonconvex optimization problems. It is known that distributed gradient-descent-type algorithms can achieve convergence to the set of global minima by adding slowly decaying Gaussian noise in order escape local minima. However, the technical assumptions under which convergence is known to occur can be restrictive in practice. In particular, in known convergence results, the local objective functions possessed by agents are required to satisfy a highly restrictive bounded-gradient-dissimilarity condition. The paper demonstrates convergence to the set of global minima while relaxing this key assumption.