Gaussian Processes with Skewed Laplace Spectral Mixture Kernels for Long-term Forecasting

TL;DR

This paper introduces a novel Gaussian process kernel based on skewed Laplace spectral mixtures, improving long-term forecasting by capturing spectral domain characteristics like skewness and heavy tails, and employs a pruning method for model simplicity.

Contribution

It proposes a new spectral mixture kernel using skewed Laplace distributions for better long-term forecasting in Gaussian processes, and adapts a pruning technique to automatically select kernel components.

Findings

Enhanced long-term forecasting accuracy with the SLSM kernel.

Robustness to the number of mixture components in spectral modeling.

Effective multivariate time series extrapolation results.

Abstract

Long-term forecasting involves predicting a horizon that is far ahead of the last observation. It is a problem of high practical relevance, for instance for companies in order to decide upon expensive long-term investments. Despite the recent progress and success of Gaussian processes (GPs) based on spectral mixture kernels, long-term forecasting remains a challenging problem for these kernels because they decay exponentially at large horizons. This is mainly due to their use of a mixture of Gaussians to model spectral densities. Characteristics of the signal important for long-term forecasting can be unravelled by investigating the distribution of the Fourier coefficients of (the training part of) the signal, which is non-smooth, heavy-tailed, sparse, and skewed. The heavy tail and skewness characteristics of such distributions in the spectral domain allow to capture long-range covariance of the signal in the time domain. Motivated by these observations, we propose to model spectral densities using a skewed Laplace spectral mixture (SLSM) due to the skewness of its peaks, sparsity, non-smoothness, and heavy tail characteristics. By applying the inverse Fourier Transform to this spectral density we obtain a new GP kernel for long-term forecasting. In addition, we adapt the lottery ticket method, originally developed to prune weights of a neural network, to GPs in order to automatically select the number of kernel components. Results of extensive experiments, including a multivariate time series, show the beneficial effect of the proposed SLSM kernel for long-term extrapolation and robustness to the choice of the number of mixture components.