A Tutorial on Distributed (Non-Bayesian) Learning: Problem, Algorithms and Results

TL;DR

This paper reviews recent advances in distributed non-Bayesian social learning algorithms, discussing their convergence properties, extensions, and applications to various network settings and hypothesis spaces.

Contribution

It provides a comprehensive overview of algorithmic solutions and theoretical results for distributed learning with finitely many hypotheses, including convergence analysis and extensions.

Findings

Convergence and rate results for non-Bayesian social learning algorithms.

Extensions to directed time-varying networks and continuum hypotheses.

Inclusion of acceleration techniques like Nesterov's method.

Abstract

We overview some results on distributed learning with focus on a family of recently proposed algorithms known as non-Bayesian social learning. We consider different approaches to the distributed learning problem and its algorithmic solutions for the case of finitely many hypotheses. The original centralized problem is discussed at first, and then followed by a generalization to the distributed setting. The results on convergence and convergence rate are presented for both asymptotic and finite time regimes. Various extensions are discussed such as those dealing with directed time-varying networks, Nesterov's acceleration technique and a continuum sets of hypothesis.