# Distributed Learning in Non-Convex Environments -- Part II: Polynomial Escape from Saddle-Points

## TL;DR

This paper demonstrates that distributed diffusion strategies can efficiently escape saddle points and reach second-order stationary points in non-convex environments, with fewer restrictions on noise conditions, advancing the understanding of distributed non-convex optimization.

## Contribution

It proves that diffusion strategies in distributed learning can escape saddle points in polynomial time under less restrictive noise assumptions, extending prior centralized results.

## Key findings

- Diffusion strategy escapes strict saddle points in O(1/μ) iterations.
- It converges to approximate second-order stationary points in polynomial time.
- Requires less restrictive conditions on gradient noise compared to prior methods.

## Abstract

The diffusion strategy for distributed learning from streaming data employs local stochastic gradient updates along with exchange of iterates over neighborhoods. In Part I [2] of this work we established that agents cluster around a network centroid and proceeded to study the dynamics of this point. We established expected descent in non-convex environments in the large-gradient regime and introduced a short-term model to examine the dynamics over finite-time horizons. Using this model, we establish in this work that the diffusion strategy is able to escape from strict saddle-points in O(1/$\mu$) iterations; it is also able to return approximately second-order stationary points in a polynomial number of iterations. Relative to prior works on the polynomial escape from saddle-points, most of which focus on centralized perturbed or stochastic gradient descent, our approach requires less restrictive conditions on the gradient noise process.