A unifying framework for generalised Bayesian online learning in non-stationary environments

TL;DR

This paper introduces BONE, a unifying probabilistic framework for online learning in non-stationary environments, enabling reinterpretation of existing methods and development of new algorithms with an open-source implementation.

Contribution

The paper presents BONE, a modular framework that unifies various online learning methods in non-stationary settings and facilitates the creation of new algorithms.

Findings

BONE can reinterpret many existing online learning methods.

The framework enables the development of novel algorithms.

Experimental results show the effectiveness of BONE-based methods.

Abstract

We propose a unifying framework for methods that perform probabilistic online learning in non-stationary environments. We call the framework BONE, which stands for generalised (B)ayesian (O)nline learning in (N)on-stationary (E)nvironments. BONE provides a common structure to tackle a variety of problems, including online continual learning, prequential forecasting, and contextual bandits. The framework requires specifying three modelling choices: (i) a model for measurements (e.g., a neural network), (ii) an auxiliary process to model non-stationarity (e.g., the time since the last changepoint), and (iii) a conditional prior over model parameters (e.g., a multivariate Gaussian). The framework also requires two algorithmic choices, which we use to carry out approximate inference under this framework: (i) an algorithm to estimate beliefs (posterior distribution) about the model parameters given the auxiliary variable, and (ii) an algorithm to estimate beliefs about the auxiliary variable. We show how the modularity of our framework allows for many existing methods to be reinterpreted as instances of BONE, and it allows us to propose new methods. We compare experimentally existing methods with our proposed new method on several datasets, providing insights into the situations that make each method more suitable for a specific task. We provide a Jax open source library to facilitate the adoption of this framework.