Stein Variational Online Changepoint Detection with Applications to Hawkes Processes and Neural Networks

TL;DR

This paper introduces SVOCD, a scalable Bayesian online changepoint detection method using Stein variational Newton, applicable to complex models like Hawkes processes and neural networks, with demonstrated real-data effectiveness.

Contribution

It extends Bayesian online changepoint detection to non-exponential family models using Stein variational Newton, enabling practical online changepoint detection in complex systems.

Findings

Effective detection in Hawkes processes

Successful application to LSTM neural networks

Demonstrated on real-world datasets

Abstract

Bayesian online changepoint detection (BOCPD) (Adams & MacKay, 2007) offers a rigorous and viable way to identify changepoints in complex systems. In this work, we introduce a Stein variational online changepoint detection (SVOCD) method to provide a computationally tractable generalization of BOCPD beyond the exponential family of probability distributions. We integrate the recently developed Stein variational Newton (SVN) method (Detommaso et al., 2018) and BOCPD to offer a full online Bayesian treatment for a large number of situations with significant importance in practice. We apply the resulting method to two challenging and novel applications: Hawkes processes and long short-term memory (LSTM) neural networks. In both cases, we successfully demonstrate the efficacy of our method on real data.