Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions

TL;DR

This paper introduces a generalized product of experts framework for combining Gaussian process predictions, achieving scalability, expressiveness, and robustness in probabilistic model fusion.

Contribution

The paper proposes a novel gPoE framework that satisfies key properties for scalable, expressive, and robust model combination, outperforming existing methods.

Findings

gPoE of Gaussian processes is highly scalable and parallelizable.

The combination method is input-dependent, enhancing expressiveness.

The approach maintains a valid probabilistic interpretation and is robust to unreliable experts.

Abstract

In this work, we propose a generalized product of experts (gPoE) framework for combining the predictions of multiple probabilistic models. We identify four desirable properties that are important for scalability, expressiveness and robustness, when learning and inferring with a combination of multiple models. Through analysis and experiments, we show that gPoE of Gaussian processes (GP) have these qualities, while no other existing combination schemes satisfy all of them at the same time. The resulting GP-gPoE is highly scalable as individual GP experts can be independently learned in parallel; very expressive as the way experts are combined depends on the input rather than fixed; the combined prediction is still a valid probabilistic model with natural interpretation; and finally robust to unreliable predictions from individual experts.