# Learning of Gaussian Processes in Distributed and Communication Limited Systems

## TL;DR

This paper investigates optimal and practical communication-efficient algorithms for learning Gaussian Processes in distributed systems, demonstrating near-optimal performance with minimal communication and outperforming existing methods.

## Contribution

It introduces an information-theoretic optimal scheme for inner-product estimation and practical quantization methods, improving distributed Gaussian Process learning.

## Key findings

- Optimal vector quantization scheme derived for inner-product estimation.
- Practical per-symbol quantization performs close to the optimal scheme.
- Proposed methods outperform existing distributed GP learning schemes with minimal communication.

## Abstract

It is of fundamental importance to find algorithms obtaining optimal performance for learning of statistical models in distributed and communication limited systems. Aiming at characterizing the optimal strategies, we consider learning of Gaussian Processes (GPs) in distributed systems as a pivotal example. We first address a very basic problem: how many bits are required to estimate the inner-products of Gaussian vectors across distributed machines? Using information theoretic bounds, we obtain an optimal solution for the problem which is based on vector quantization. Two suboptimal and more practical schemes are also presented as substitute for the vector quantization scheme. In particular, it is shown that the performance of one of the practical schemes which is called per-symbol quantization is very close to the optimal one. Schemes provided for the inner-product calculations are incorporated into our proposed distributed learning methods for GPs. Experimental results show that with spending few bits per symbol in our communication scheme, our proposed methods outperform previous zero rate distributed GP learning schemes such as Bayesian Committee Model (BCM) and Product of experts (PoE).