Adaptive Sparse Gaussian Process

TL;DR

This paper introduces an adaptive sparse Gaussian Process algorithm that efficiently updates the model in non-stationary environments by updating a single inducing point, enabling fast online inference with minimal computational cost.

Contribution

It reformulates a variational sparse GP with a forgetting factor and simplifies inference by updating only one inducing point per new sample, enhancing adaptability and efficiency.

Findings

Fast convergence of inference process

Effective modeling of non-stationary data

Outperforms state-of-the-art approaches in predictive accuracy

Abstract

Adaptive learning is necessary for non-stationary environments where the learning machine needs to forget past data distribution. Efficient algorithms require a compact model update to not grow in computational burden with the incoming data and with the lowest possible computational cost for online parameter updating. Existing solutions only partially cover these needs. Here, we propose the first adaptive sparse Gaussian Process (GP) able to address all these issues. We first reformulate a variational sparse GP algorithm to make it adaptive through a forgetting factor. Next, to make the model inference as simple as possible, we propose updating a single inducing point of the sparse GP model together with the remaining model parameters every time a new sample arrives. As a result, the algorithm presents a fast convergence of the inference process, which allows an efficient model update (with a single inference iteration) even in highly non-stationary environments. Experimental results demonstrate the capabilities of the proposed algorithm and its good performance in modeling the predictive posterior in mean and confidence interval estimation compared to state-of-the-art approaches.