The Importance of Robust Features in Mitigating Catastrophic Forgetting

TL;DR

This paper investigates how training continual learning models on robust features can reduce catastrophic forgetting, showing that robust features help preserve previously learned knowledge.

Contribution

It introduces the CL robust dataset and demonstrates that models trained on it experience less catastrophic forgetting compared to standard datasets.

Findings

Models trained on CL robust dataset show reduced forgetting.

Robust features significantly improve continual learning performance.

Study highlights the importance of feature robustness in CL.

Abstract

Continual learning (CL) is an approach to address catastrophic forgetting, which refers to forgetting previously learned knowledge by neural networks when trained on new tasks or data distributions. The adversarial robustness has decomposed features into robust and non-robust types and demonstrated that models trained on robust features significantly enhance adversarial robustness. However, no study has been conducted on the efficacy of robust features from the lens of the CL model in mitigating catastrophic forgetting in CL. In this paper, we introduce the CL robust dataset and train four baseline models on both the standard and CL robust datasets. Our results demonstrate that the CL models trained on the CL robust dataset experienced less catastrophic forgetting of the previously learned tasks than when trained on the standard dataset. Our observations highlight the significance of the features provided to the underlying CL models, showing that CL robust features can alleviate catastrophic forgetting.