Transfer without Forgetting

TL;DR

This paper introduces Transfer without Forgetting (TwF), a hybrid method that leverages a fixed pretrained network to improve continual learning by mitigating catastrophic forgetting and enhancing knowledge transfer.

Contribution

We propose TwF, a novel hybrid approach that maintains a fixed pretrained network to facilitate continual learning without forgetting, outperforming existing methods.

Findings

TwF achieves a 4.81% average gain in Class-Incremental accuracy.

TwF outperforms other continual learning methods across various datasets.

The approach effectively mitigates catastrophic forgetting during transfer learning.

Abstract

This work investigates the entanglement between Continual Learning (CL) and Transfer Learning (TL). In particular, we shed light on the widespread application of network pretraining, highlighting that it is itself subject to catastrophic forgetting. Unfortunately, this issue leads to the under-exploitation of knowledge transfer during later tasks. On this ground, we propose Transfer without Forgetting (TwF), a hybrid approach building upon a fixed pretrained sibling network, which continuously propagates the knowledge inherent in the source domain through a layer-wise loss term. Our experiments indicate that TwF steadily outperforms other CL methods across a variety of settings, averaging a 4.81% gain in Class-Incremental accuracy over a variety of datasets and different buffer sizes.