Learning spectrograms with convolutional spectral kernels

TL;DR

This paper introduces the convolutional spectral kernel (CSK), a new non-stationary covariance kernel for Gaussian processes that enhances modeling of non-stationary patterns and improves generalization on spatiotemporal data.

Contribution

The paper proposes the CSK, a novel non-stationary kernel derived from convolution of radial basis functions, with a framework for interpretability and scalable inference methods.

Findings

Improved generalization on spatiotemporal datasets

Ability to extract non-stationary periodic patterns

Enhanced interpretability through spectrogram analysis

Abstract

We introduce the convolutional spectral kernel (CSK), a novel family of non-stationary, nonparametric covariance kernels for Gaussian process (GP) models, derived from the convolution between two imaginary radial basis functions. We present a principled framework to interpret CSK, as well as other deep probabilistic models, using approximated Fourier transform, yielding a concise representation of input-frequency spectrogram. Observing through the lens of the spectrogram, we provide insight on the interpretability of deep models. We then infer the functional hyperparameters using scalable variational and MCMC methods. On small- and medium-sized spatiotemporal datasets, we demonstrate improved generalization of GP models when equipped with CSK, and their capability to extract non-stationary periodic patterns.