# Linear Multiple Low-Rank Kernel Based Stationary Gaussian Processes Regression for Time Series

## TL;DR

This paper introduces a novel grid spectral mixture kernel for Gaussian process regression in time series, enabling efficient hyper-parameter estimation and outperforming existing methods in prediction accuracy and stability.

## Contribution

The paper proposes a new GSM kernel approximating stationary kernels with a linear combination of low-rank sub-kernels, along with efficient optimization methods for hyper-parameters.

## Key findings

- GSM kernel achieves close approximation of stationary kernels.
- The proposed methods outperform competitors in prediction MSE.
- Experimental results show improved numerical stability.

## Abstract

Gaussian processes (GP) for machine learning have been studied systematically over the past two decades and they are by now widely used in a number of diverse applications. However, GP kernel design and the associated hyper-parameter optimization are still hard and to a large extend open problems. In this paper, we consider the task of GP regression for time series modeling and analysis. The underlying stationary kernel can be approximated arbitrarily close by a new proposed grid spectral mixture (GSM) kernel, which turns out to be a linear combination of low-rank sub-kernels. In the case where a large number of the sub-kernels are used, either the Nystr\"{o}m or the random Fourier feature approximations can be adopted to deal efficiently with the computational demands. The unknown GP hyper-parameters consist of the non-negative weights of all sub-kernels as well as the noise variance; their estimation is performed via the maximum-likelihood (ML) estimation framework. Two efficient numerical optimization methods for solving the unknown hyper-parameters are derived, including a sequential majorization-minimization (MM) method and a non-linearly constrained alternating direction of multiplier method (ADMM). The MM matches perfectly with the proven low-rank property of the proposed GSM sub-kernels and turns out to be a part of efficiency, stable, and efficient solver, while the ADMM has the potential to generate better local minimum in terms of the test MSE. Experimental results, based on various classic time series data sets, corroborate that the proposed GSM kernel-based GP regression model outperforms several salient competitors of similar kind in terms of prediction mean-squared-error and numerical stability.