Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates

TL;DR

This paper introduces FOO-VB, a novel online variational Bayes algorithm with fixed-point updates for task-agnostic continual learning, effectively mitigating catastrophic forgetting without external memory or task boundary knowledge.

Contribution

We derive fixed-point equations for online variational Bayes with Gaussian distributions and develop FOO-VB, a new method for continual learning in unknown task boundary scenarios.

Findings

FOO-VB outperforms existing methods in task-agnostic continual learning.

The fixed-point equations enable accurate online Bayesian updates.

The method does not require external memory or prior task information.

Abstract

Background: Catastrophic forgetting is the notorious vulnerability of neural networks to the changes in the data distribution during learning. This phenomenon has long been considered a major obstacle for using learning agents in realistic continual learning settings. A large body of continual learning research assumes that task boundaries are known during training. However, only a few works consider scenarios in which task boundaries are unknown or not well defined -- task agnostic scenarios. The optimal Bayesian solution for this requires an intractable online Bayes update to the weights posterior. Contributions: We aim to approximate the online Bayes update as accurately as possible. To do so, we derive novel fixed-point equations for the online variational Bayes optimization problem, for multivariate Gaussian parametric distributions. By iterating the posterior through these fixed-point equations, we obtain an algorithm (FOO-VB) for continual learning which can handle non-stationary data distribution using a fixed architecture and without using external memory (i.e. without access to previous data). We demonstrate that our method (FOO-VB) outperforms existing methods in task agnostic scenarios. FOO-VB Pytorch implementation will be available online.