Stationarity without mean reversion in improper Gaussian processes

TL;DR

This paper introduces a new class of stationary Gaussian process priors with infinite variance that are not mean reverting, addressing limitations of traditional stationary kernels in GP regression.

Contribution

It proposes improper Gaussian process priors with non-positive kernels that are stationary but non-reverting, expanding the modeling capabilities of GP regression.

Findings

Non-positive kernels can define stationary, non-mean reverting processes.

Analytical posterior formulas are derived for these processes.

Synthetic and real data show improved modeling of non-reverting behaviors.

Abstract

The behavior of a GP regression depends on the choice of covariance function. Stationary covariance functions are preferred in machine learning applications. However, (non-periodic) stationary covariance functions are always mean reverting and can therefore exhibit pathological behavior when applied to data that does not relax to a fixed global mean value. In this paper we show that it is possible to use improper GP priors with infinite variance to define processes that are stationary but not mean reverting. To this aim, we use of non-positive kernels that can only be defined in this limit regime. The resulting posterior distributions can be computed analytically and it involves a simple correction of the usual formulas. The main contribution of the paper is the introduction of a large family of smooth non-reverting covariance functions that closely resemble the kernels commonly used in the GP literature (e.g. squared exponential and Mat\'ern class). By analyzing both synthetic and real data, we demonstrate that these non-positive kernels solve some known pathologies of mean reverting GP regression while retaining most of the favorable properties of ordinary smooth stationary kernels.