# Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition

## TL;DR

This paper introduces a probabilistic deep autoencoder framework that enhances continual learning by effectively distinguishing unknown data and reducing catastrophic forgetting through variational inference and generative replay.

## Contribution

It presents a novel unified approach combining open set recognition and continual learning using variational inference in a deep autoencoder, addressing both unknown data detection and knowledge retention.

## Key findings

- Effective detection of out-of-distribution data.
- Significant reduction in catastrophic interference.
- Improved robustness in continual learning scenarios.

## Abstract

Modern deep neural networks are well known to be brittle in the face of unknown data instances and recognition of the latter remains a challenge. Although it is inevitable for continual-learning systems to encounter such unseen concepts, the corresponding literature appears to nonetheless focus primarily on alleviating catastrophic interference with learned representations. In this work, we introduce a probabilistic approach that connects these perspectives based on variational inference in a single deep autoencoder model. Specifically, we propose to bound the approximate posterior by fitting regions of high density on the basis of correctly classified data points. These bounds are shown to serve a dual purpose: unseen unknown out-of-distribution data can be distinguished from already trained known tasks towards robust application. Simultaneously, to retain already acquired knowledge, a generative replay process can be narrowed to strictly in-distribution samples, in order to significantly alleviate catastrophic interference.