Continual Learning with Neuron Activation Importance

TL;DR

This paper introduces a neuron activation importance-based regularization technique to enhance stability and robustness in continual learning, enabling models to learn sequential tasks without forgetting previous knowledge.

Contribution

It proposes a novel regularization method based on neuron activation importance for stable and order-agnostic continual learning.

Findings

Improved classification accuracy on benchmark datasets.

Enhanced stability and plasticity in continual learning.

Robust performance across different task sequences.

Abstract

Continual learning is a concept of online learning with multiple sequential tasks. One of the critical barriers of continual learning is that a network should learn a new task keeping the knowledge of old tasks without access to any data of the old tasks. In this paper, we propose a neuron activation importance-based regularization method for stable continual learning regardless of the order of tasks. We conduct comprehensive experiments on existing benchmark data sets to evaluate not just the stability and plasticity of our method with improved classification accuracy also the robustness of the performance along the changes of task order.