BAM: Bayes with Adaptive Memory

TL;DR

BAM introduces a flexible Bayesian framework that selectively remembers or forgets past data, enabling agents to adapt effectively in non-stationary environments with changing conditions.

Contribution

The paper proposes Bayes with Adaptive Memory (BAM), a novel approach allowing selective retention of past observations to improve adaptation in dynamic settings.

Findings

BAM outperforms traditional Bayesian methods in non-stationary environments.

It generalizes many existing Bayesian update rules.

Experimental results show continuous adaptation in changing scenarios.

Abstract

Online learning via Bayes' theorem allows new data to be continuously integrated into an agent's current beliefs. However, a naive application of Bayesian methods in non stationary environments leads to slow adaptation and results in state estimates that may converge confidently to the wrong parameter value. A common solution when learning in changing environments is to discard/downweight past data; however, this simple mechanism of "forgetting" fails to account for the fact that many real-world environments involve revisiting similar states. We propose a new framework, Bayes with Adaptive Memory (BAM), that takes advantage of past experience by allowing the agent to choose which past observations to remember and which to forget. We demonstrate that BAM generalizes many popular Bayesian update rules for non-stationary environments. Through a variety of experiments, we demonstrate the ability of BAM to continuously adapt in an ever-changing world.