Localizing Catastrophic Forgetting in Neural Networks

TL;DR

This paper introduces a method to identify which parameters in neural networks contribute most to catastrophic forgetting, aiding understanding and mitigation of this issue in continual learning scenarios.

Contribution

It presents a novel approach for localizing parameter contributions to catastrophic forgetting in neural networks, filling a gap in recent research.

Findings

Method successfully identifies key parameters involved in forgetting.

Analysis across three continual learning scenarios demonstrates the method's effectiveness.

Provides insights into parameter roles during sequential task learning.

Abstract

Artificial neural networks (ANNs) suffer from catastrophic forgetting when trained on a sequence of tasks. While this phenomenon was studied in the past, there is only very limited recent research on this phenomenon. We propose a method for determining the contribution of individual parameters in an ANN to catastrophic forgetting. The method is used to analyze an ANNs response to three different continual learning scenarios.