Network Independent Rates in Distributed Learning

TL;DR

This paper introduces a new belief update rule for distributed learning over dynamic directed networks, achieving network-independent convergence rates, which were previously unavailable for similar algorithms.

Contribution

The paper presents a novel belief update rule inspired by Push-Sum, providing explicit convergence rate characterization and achieving network-independent belief concentration.

Findings

Agents reach consensus at a network-independent rate

The update rule is consistent and effective in time-varying directed graphs

Explicit convergence rate characterization is provided

Abstract

We propose a new belief update rule for Distributed Non-Bayesian learning in time-varying directed graphs, where a group of agents tries to collectively identify a hypothesis that best describes a sequence of observed data. We show that the proposed update rule, inspired by the Push-Sum algorithm, is consistent, moreover we provide an explicit characterization of its convergence rate. Our main result states that, after a transient time, all agents will concentrate their beliefs at a network independent rate. Network independent rates were not available for other consensus based distributed learning algorithms.