Differentially Private Continual Learning

TL;DR

This paper introduces a differentially private continual learning framework that uses generative models and variational inference to mitigate catastrophic forgetting while respecting data privacy constraints.

Contribution

It proposes a novel approach combining differential privacy with variational inference for continual learning, enabling models to remember old data without compromising privacy.

Findings

Effective mitigation of catastrophic forgetting.

Maintains privacy of old datasets.

Compatible with existing neural network architectures.

Abstract

Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons. For example, hospitals might not be able to retain patient data permanently. But neural networks trained on recent data alone will tend to forget lessons learned on old data. We present a differentially private continual learning framework based on variational inference. We estimate the likelihood of past data given the current model using differentially private generative models of old datasets.