Decentralized Online Ensembles of Gaussian Processes for Multi-Agent Systems

TL;DR

This paper presents a fully decentralized method for Gaussian process approximation in multi-agent systems, utilizing online Bayesian model averaging to improve hyperparameter selection and ensemble performance.

Contribution

It introduces a novel decentralized algorithm for Gaussian process approximation and an ensemble scheme for Bayesian multiple kernel learning in multi-agent settings.

Findings

The method achieves asymptotic exactness in decentralized Gaussian process approximation.

Ensembling improves hyperparameter selection and model accuracy.

The approach outperforms existing Bayesian and frequentist methods on various datasets.

Abstract

Flexible and scalable decentralized learning solutions are fundamentally important in the application of multi-agent systems. While several recent approaches introduce (ensembles of) kernel machines in the distributed setting, Bayesian solutions are much more limited. We introduce a fully decentralized, asymptotically exact solution to computing the random feature approximation of Gaussian processes. We further address the choice of hyperparameters by introducing an ensembling scheme for Bayesian multiple kernel learning based on online Bayesian model averaging. The resulting algorithm is tested against Bayesian and frequentist methods on simulated and real-world datasets.