Neural Non-Stationary Spectral Kernel

TL;DR

This paper introduces a neural network-based approach to model non-stationary spectral kernels in Gaussian processes, demonstrating improved performance over traditional stationary kernels on benchmark datasets.

Contribution

The paper proposes using neural networks to model hyperparameter functions in non-stationary spectral kernels, enhancing flexibility and performance in Gaussian process modeling.

Findings

Neural non-stationary spectral kernels outperform stationary kernels on benchmarks.

The neural variant achieves the best performance among tested methods.

Scalable Gaussian process inference is effectively implemented with the proposed kernels.

Abstract

Standard kernels such as Mat\'ern or RBF kernels only encode simple monotonic dependencies within the input space. Spectral mixture kernels have been proposed as general-purpose, flexible kernels for learning and discovering more complicated patterns in the data. Spectral mixture kernels have recently been generalized into non-stationary kernels by replacing the mixture weights, frequency means and variances by input-dependent functions. These functions have also been modelled as Gaussian processes on their own. In this paper we propose modelling the hyperparameter functions with neural networks, and provide an experimental comparison between the stationary spectral mixture and the two non-stationary spectral mixtures. Scalable Gaussian process inference is implemented within the sparse variational framework for all the kernels considered. We show that the neural variant of the kernel is able to achieve the best performance, among alternatives, on several benchmark datasets.