Distributed multi-agent Gaussian regression via finite-dimensional approximations

TL;DR

This paper introduces scalable distributed Gaussian process estimation methods for multi-agent systems using finite-dimensional approximations, reducing computational and communication costs while maintaining accuracy.

Contribution

It proposes two novel suboptimal approaches based on eigenfunction approximations, with probabilistic bounds and adaptive tuning strategies for efficient distributed Gaussian regression.

Findings

Methods scale with eigenfunction dimension E, not total data M.

Probabilistic bounds guide desired estimation accuracy.

Algorithms validated on synthetic and real data.

Abstract

We consider the problem of distributedly estimating Gaussian processes in multi-agent frameworks. Each agent collects few measurements and aims to collaboratively reconstruct a common estimate based on all data. Agents are assumed with limited computational and communication capabilities and to gather $M$ noisy measurements in total on input locations independently drawn from a known common probability density. The optimal solution would require agents to exchange all the $M$ input locations and measurements and then invert an $M \times M$ matrix, a non-scalable task. Differently, we propose two suboptimal approaches using the first $E$ orthonormal eigenfunctions obtained from the \ac{KL} expansion of the chosen kernel, where typically $E \ll M$. The benefits are that the computation and communication complexities scale with $E$ and not with $M$, and computing the required statistics can be performed via standard average consensus algorithms. We obtain probabilistic non-asymptotic bounds that determine a priori the desired level of estimation accuracy, and new distributed strategies relying on Stein's unbiased risk estimate (SURE) paradigms for tuning the regularization parameters and applicable to generic basis functions (thus not necessarily kernel eigenfunctions) and that can again be implemented via average consensus. The proposed estimators and bounds are finally tested on both synthetic and real field data.