The Minecraft Kernel: Modelling correlated Gaussian Processes in the Fourier domain

TL;DR

This paper introduces a novel multi-output Gaussian process kernel in the Fourier domain that overcomes limitations of spectral mixture kernels, enabling more accurate modeling of cross-covariances across multiple outputs.

Contribution

It proposes a new family of kernels using block components to generalize spectral mixture kernels for multi-output Gaussian processes, allowing arbitrary stationary kernel approximation.

Findings

First multi-output spectral mixture kernel capable of arbitrary approximation

Addresses the blind spot in cross-covariance modeling

Enables more flexible and accurate multi-output Gaussian process modeling

Abstract

In the univariate setting, using the kernel spectral representation is an appealing approach for generating stationary covariance functions. However, performing the same task for multiple-output Gaussian processes is substantially more challenging. We demonstrate that current approaches to modelling cross-covariances with a spectral mixture kernel possess a critical blind spot. For a given pair of processes, the cross-covariance is not reproducible across the full range of permitted correlations, aside from the special case where their spectral densities are of identical shape. We present a solution to this issue by replacing the conventional Gaussian components of a spectral mixture with block components of finite bandwidth (i.e. rectangular step functions). The proposed family of kernel represents the first multi-output generalisation of the spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.