Context-tree weighting for real-valued time series: Bayesian inference with hierarchical mixture models

TL;DR

This paper introduces a hierarchical Bayesian framework using context trees for modeling real-valued time series, enabling flexible mixture models and efficient inference, demonstrated with autoregressive models outperforming existing methods.

Contribution

It develops a novel hierarchical Bayesian approach with context trees for real-valued time series, allowing flexible mixture modeling and exact inference, applicable with various model classes.

Findings

Outperforms state-of-the-art techniques on simulated data

Provides a flexible framework for mixture modeling of time series

Enables efficient, exact Bayesian inference in hierarchical models

Abstract

Real-valued time series are ubiquitous in the sciences and engineering. In this work, a general, hierarchical Bayesian modelling framework is developed for building mixture models for times series. This development is based, in part, on the use of context trees, and it includes a collection of effective algorithmic tools for learning and inference. A discrete context (or 'state') is extracted for each sample, consisting of a discretised version of some of the most recent observations preceding it. The set of all relevant contexts are represented as a discrete context-tree. At the bottom level, a different real-valued time series model is associated with each context-state, i.e., with each leaf of the tree. This defines a very general framework that can be used in conjunction with any existing model class to build flexible and interpretable mixture models. Extending the idea of context-tree weighting leads to algorithms that allow for efficient, exact Bayesian inference in this setting. The utility of the general framework is illustrated in detail when autoregressive (AR) models are used at the bottom level, resulting in a nonlinear AR mixture model. The associated methods are found to outperform several state-of-the-art techniques on simulated and real-world experiments.