Variational Continual Learning

TL;DR

This paper introduces Variational Continual Learning (VCL), a framework combining online variational inference and Monte Carlo VI to enable neural networks to learn sequentially without forgetting previous tasks.

Contribution

VCL is a novel, general framework that effectively trains deep models in evolving continual learning environments, outperforming existing methods.

Findings

VCL outperforms state-of-the-art continual learning methods.

VCL effectively prevents catastrophic forgetting.

VCL works for both discriminative and generative models.

Abstract

This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in complex continual learning settings where existing tasks evolve over time and entirely new tasks emerge. Experimental results show that VCL outperforms state-of-the-art continual learning methods on a variety of tasks, avoiding catastrophic forgetting in a fully automatic way.