Adversarial Feature Alignment: Avoid Catastrophic Forgetting in Incremental Task Lifelong Learning

TL;DR

This paper introduces an adversarial feature alignment approach for incremental multi-task image classification that effectively reduces catastrophic forgetting and outperforms existing methods in lifelong learning scenarios.

Contribution

It proposes a novel adversarial feature alignment method that uses multi-stage feature guidance to mitigate forgetting in lifelong learning, inspired by human learning processes.

Findings

Outperforms state-of-the-art methods in accuracy on new tasks.

Achieves better retention of old task performance.

Provides a scalable solution for incremental multi-task learning.

Abstract

Human beings are able to master a variety of knowledge and skills with ongoing learning. By contrast, dramatic performance degradation is observed when new tasks are added to an existing neural network model. This phenomenon, termed as \emph{Catastrophic Forgetting}, is one of the major roadblocks that prevent deep neural networks from achieving human-level artificial intelligence. Several research efforts, e.g. \emph{Lifelong} or \emph{Continual} learning algorithms, have been proposed to tackle this problem. However, they either suffer from an accumulating drop in performance as the task sequence grows longer, or require to store an excessive amount of model parameters for historical memory, or cannot obtain competitive performance on the new tasks. In this paper, we focus on the incremental multi-task image classification scenario. Inspired by the learning process of human students, where they usually decompose complex tasks into easier goals, we propose an adversarial feature alignment method to avoid catastrophic forgetting. In our design, both the low-level visual features and high-level semantic features serve as soft targets and guide the training process in multiple stages, which provide sufficient supervised information of the old tasks and help to reduce forgetting. Due to the knowledge distillation and regularization phenomenons, the proposed method gains even better performance than finetuning on the new tasks, which makes it stand out from other methods. Extensive experiments in several typical lifelong learning scenarios demonstrate that our method outperforms the state-of-the-art methods in both accuracies on new tasks and performance preservation on old tasks.