Gaussian Experts Selection using Graphical Models

TL;DR

This paper introduces a method for selecting Gaussian process experts using graphical models to balance computational efficiency and accurate uncertainty quantification in large-scale GP approximations.

Contribution

It proposes a theory-guided expert selection approach leveraging graphical models and sparse precision matrices to improve efficiency without sacrificing uncertainty calibration.

Findings

Reduces computational cost by eliminating weak experts.

Maintains calibrated uncertainty quantification.

Enhances prediction accuracy with dependent expert modeling.

Abstract

Local approximations are popular methods to scale Gaussian processes (GPs) to big data. Local approximations reduce time complexity by dividing the original dataset into subsets and training a local expert on each subset. Aggregating the experts' prediction is done assuming either conditional dependence or independence between the experts. Imposing the \emph{conditional independence assumption} (CI) between the experts renders the aggregation of different expert predictions time efficient at the cost of poor uncertainty quantification. On the other hand, modeling dependent experts can provide precise predictions and uncertainty quantification at the expense of impractically high computational costs. By eliminating weak experts via a theory-guided expert selection step, we substantially reduce the computational cost of aggregating dependent experts while ensuring calibrated uncertainty quantification. We leverage techniques from the literature on undirected graphical models, using sparse precision matrices that encode conditional dependencies between experts to select the most important experts. Moreov