Doubly Robust Bayesian Inference for Non-Stationary Streaming Data with $\beta$-Divergences

TL;DR

This paper introduces a robust Bayesian online changepoint detection method using $eta$-divergences within General Bayesian Inference, achieving doubly robust, scalable, and state-of-the-art performance on real-world data.

Contribution

It develops the first robust Bayesian online changepoint detection algorithm with $eta$-divergences, scalable GBI approximations, and an online method for selecting the divergence parameter.

Findings

Reduces false discovery rates of changepoints from over 90% to 0%.

Demonstrates linear time and constant space complexity.

Validates on Bayesian Linear Regression models.

Abstract

We present the very first robust Bayesian Online Changepoint Detection algorithm through General Bayesian Inference (GBI) with $\beta$-divergences. The resulting inference procedure is doubly robust for both the parameter and the changepoint (CP) posterior, with linear time and constant space complexity. We provide a construction for exponential models and demonstrate it on the Bayesian Linear Regression model. In so doing, we make two additional contributions: Firstly, we make GBI scalable using Structural Variational approximations that are exact as $\beta \to 0$. Secondly, we give a principled way of choosing the divergence parameter $\beta$ by minimizing expected predictive loss on-line. Reducing False Discovery Rates of CPs from more than 90% to 0% on real world data, this offers the state of the art.