The Hierarchical Adaptive Forgetting Variational Filter

TL;DR

This paper introduces a hierarchical Bayesian variational filter that adaptively detects distribution changes in streaming data, with applications in reinforcement learning, Bayesian autoregressive modeling, and stochastic gradient descent optimization.

Contribution

It presents a novel hierarchical Bayesian algorithm with variational updates for models from the exponential family, linking it to reinforcement learning and adaptive data stream analysis.

Findings

Effective in detecting distribution shifts in data streams

Applicable to reinforcement learning and Bayesian autoregressive models

Enhances stochastic gradient descent optimization

Abstract

A common problem in Machine Learning and statistics consists in detecting whether the current sample in a stream of data belongs to the same distribution as previous ones, is an isolated outlier or inaugurates a new distribution of data. We present a hierarchical Bayesian algorithm that aims at learning a time-specific approximate posterior distribution of the parameters describing the distribution of the data observed. We derive the update equations of the variational parameters of the approximate posterior at each time step for models from the exponential family, and show that these updates find interesting correspondents in Reinforcement Learning (RL). In this perspective, our model can be seen as a hierarchical RL algorithm that learns a posterior distribution according to a certain stability confidence that is, in turn, learned according to its own stability confidence. Finally, we show some applications of our generic model, first in a RL context, next with an adaptive Bayesian Autoregressive model, and finally in the context of Stochastic Gradient Descent optimization.