Shared and Private VAEs with Generative Replay for Continual Learning

TL;DR

This paper introduces a hybrid variational autoencoder model with shared and task-specific components, combining generative replay and architectural growth to improve continual learning and prevent forgetting in neural networks.

Contribution

It proposes a novel hybrid VAE architecture with shared and task-specific parts, effectively addressing catastrophic forgetting in continual learning scenarios.

Findings

Achieves state-of-the-art results on multiple visual benchmarks.

Effectively prevents catastrophic forgetting in continual learning.

Demonstrates robustness across diverse datasets.

Abstract

Continual learning tries to learn new tasks without forgetting previously learned ones. In reality, most of the existing artificial neural network(ANN) models fail, while humans do the same by remembering previous works throughout their life. Although simply storing all past data can alleviate the problem, it needs large memory and often infeasible in real-world applications where last data access is limited. We hypothesize that the model that learns to solve each task continually has some task-specific properties and some task-invariant characteristics. We propose a hybrid continual learning model that is more suitable in real case scenarios to address the issues that has a task-invariant shared variational autoencoder and T task-specific variational autoencoders. Our model combines generative replay and architectural growth to prevent catastrophic forgetting. We show our hybrid model effectively avoids forgetting and achieves state-of-the-art results on visual continual learning benchmarks such as MNIST, Permuted MNIST(QMNIST), CIFAR100, and miniImageNet datasets. We discuss results on a few more datasets, such as SVHN, Fashion-MNIST, EMNIST, and CIFAR10.