Additive Gaussian Processes Revisited

TL;DR

This paper introduces the orthogonal additive kernel (OAK) for Gaussian Processes, enhancing interpretability and efficiency by imposing orthogonality constraints, and demonstrates its competitive predictive performance with fewer additive terms.

Contribution

The paper proposes the OAK kernel, which enforces orthogonality in additive Gaussian Processes, improving identifiability, interpretability, and convergence rates compared to previous models.

Findings

OAK achieves similar or better predictive accuracy with fewer terms.

Improved convergence rates for sparse computation methods.

Enhanced interpretability through orthogonal additive structure.

Abstract

Gaussian Process (GP) models are a class of flexible non-parametric models that have rich representational power. By using a Gaussian process with additive structure, complex responses can be modelled whilst retaining interpretability. Previous work showed that additive Gaussian process models require high-dimensional interaction terms. We propose the orthogonal additive kernel (OAK), which imposes an orthogonality constraint on the additive functions, enabling an identifiable, low-dimensional representation of the functional relationship. We connect the OAK kernel to functional ANOVA decomposition, and show improved convergence rates for sparse computation methods. With only a small number of additive low-dimensional terms, we demonstrate the OAK model achieves similar or better predictive performance compared to black-box models, while retaining interpretability.