Bridging Theory and Practice in Efficient Gaussian Process-Based Statistical Modeling for Large Datasets

TL;DR

This paper introduces the piecewise continuous Gaussian process (PCGP), a scalable and efficient alternative to traditional GPs for large datasets, addressing computational challenges while maintaining probabilistic richness.

Contribution

The paper proposes the PCGP, a novel process that overcomes limitations of existing scalable GPs by preserving probabilistic structure and improving computational efficiency.

Findings

PCGP offers substantial computational efficiency improvements.

PCGP retains the probabilistic richness of traditional GPs.

Numerical illustrations demonstrate PCGP's effectiveness.

Abstract

Geostatistics is a branch of statistics concerned with stochastic processes over continuous domains, with Gaussian processes (GPs) providing a flexible and principled modelling framework. However, the high computational cost of simulating or computing likelihoods with GPs limits their scalability to large datasets. This paper introduces the piecewise continuous Gaussian process (PCGP), a new process that retains the rich probabilistic structure of traditional GPs while offering substantial computational efficiency. As will be shown and discussed, existing scalable approaches that define stochastic processes on continuous domains -- such as the nearest-neighbour GP (NNGP) and the radial-neighbour GP (RNGP) -- rely on conditional independence structures that effectively constrain the measurable space on which the processes are defined, which may induce undesirable probabilistic behaviour and compromise their practical applicability, particularly in complex latent GP models. The PCGP mitigates these limitations and provides a theoretically grounded and computationally efficient alternative, as demonstrated through numerical illustrations.