Collective Online Learning of Gaussian Processes in Massive Multi-Agent Systems

TL;DR

This paper introduces a decentralized Gaussian Process framework enabling multiple agents in large distributed systems to collaboratively learn a global model efficiently through peer-to-peer communication, avoiding central bottlenecks.

Contribution

It proposes a novel collective online learning approach for Gaussian Processes that enhances scalability and robustness in massive multi-agent systems.

Findings

Demonstrates improved scalability over centralized methods

Shows effective convergence on synthetic datasets

Validates performance on real-world data

Abstract

Distributed machine learning (ML) is a modern computation paradigm that divides its workload into independent tasks that can be simultaneously achieved by multiple machines (i.e., agents) for better scalability. However, a typical distributed system is usually implemented with a central server that collects data statistics from multiple independent machines operating on different subsets of data to build a global analytic model. This centralized communication architecture however exposes a single choke point for operational failure and places severe bottlenecks on the server's communication and computation capacities as it has to process a growing volume of communication from a crowd of learning agents. To mitigate these bottlenecks, this paper introduces a novel Collective Online Learning Gaussian Process framework for massive distributed systems that allows each agent to build its local model, which can be exchanged and combined efficiently with others via peer-to-peer communication to converge on a global model of higher quality. Finally, our empirical results consistently demonstrate the efficiency of our framework on both synthetic and real-world datasets.