BayGo: Joint Bayesian Learning and Information-Aware Graph Optimization

TL;DR

BayGo introduces a decentralized framework for multi-agent Bayesian learning that optimizes communication and graph structure, leading to faster convergence and improved accuracy without prior data knowledge.

Contribution

It presents a novel joint Bayesian learning and graph optimization method that ensures fast convergence and minimal communication in decentralized multi-agent systems.

Findings

Faster convergence compared to traditional topologies

Higher accuracy in learning outcomes

Exponential decrease in estimation error

Abstract

This article deals with the problem of distributed machine learning, in which agents update their models based on their local datasets, and aggregate the updated models collaboratively and in a fully decentralized manner. In this paper, we tackle the problem of information heterogeneity arising in multi-agent networks where the placement of informative agents plays a crucial role in the learning dynamics. Specifically, we propose BayGo, a novel fully decentralized joint Bayesian learning and graph optimization framework with proven fast convergence over a sparse graph. Under our framework, agents are able to learn and communicate with the most informative agent to their own learning. Unlike prior works, our framework assumes no prior knowledge of the data distribution across agents nor does it assume any knowledge of the true parameter of the system. The proposed alternating minimization based framework ensures global connectivity in a fully decentralized way while minimizing the number of communication links. We theoretically show that by optimizing the proposed objective function, the estimation error of the posterior probability distribution decreases exponentially at each iteration. Via extensive simulations, we show that our framework achieves faster convergence and higher accuracy compared to fully-connected and star topology graphs.