Peer-to-peer Federated Learning on Graphs

TL;DR

This paper introduces a decentralized Bayesian-like federated learning algorithm for graph-structured networks, enabling nodes to collaboratively learn models with improved accuracy through neighbor information sharing.

Contribution

It proposes a novel distributed belief update algorithm for federated learning on graphs, with theoretical error bounds and practical approximations for deep neural network training.

Findings

Significant accuracy improvement over non-cooperative learning.

Effective belief aggregation from one-hop neighbors.

Applicable to both linear models and deep neural networks.

Abstract

We consider the problem of training a machine learning model over a network of nodes in a fully decentralized framework. The nodes take a Bayesian-like approach via the introduction of a belief over the model parameter space. We propose a distributed learning algorithm in which nodes update their belief by aggregate information from their one-hop neighbors to learn a model that best fits the observations over the entire network. In addition, we also obtain sufficient conditions to ensure that the probability of error is small for every node in the network. We discuss approximations required for applying this algorithm to train Deep Neural Networks (DNNs). Experiments on training linear regression model and on training a DNN show that the proposed learning rule algorithm provides a significant improvement in the accuracy compared to the case where nodes learn without cooperation.