Incremental Classifier Learning Based on PEDCC-Loss and Cosine Distance

TL;DR

This paper proposes an ensemble incremental classifier method using PEDCC-Loss and cosine distance to effectively mitigate catastrophic forgetting in neural networks during incremental learning.

Contribution

It introduces a novel ensemble approach with PEDCC-Loss and cosine distance, preserving old knowledge without relying on old samples or knowledge distillation.

Findings

Outperforms LwF and iCaRL methods on EMINST and CIFAR100 datasets.

Effectively mitigates catastrophic forgetting in incremental learning.

Preserves old knowledge while learning new classes.

Abstract

The main purpose of incremental learning is to learn new knowledge while not forgetting the knowledge which have been learned before. At present, the main challenge in this area is the catastrophe forgetting, namely the network will lose their performance in the old tasks after training for new tasks. In this paper, we introduce an ensemble method of incremental classifier to alleviate this problem, which is based on the cosine distance between the output feature and the pre-defined center, and can let each task to be preserved in different networks. During training, we make use of PEDCC-Loss to train the CNN network. In the stage of testing, the prediction is determined by the cosine distance between the network latent features and pre-defined center. The experimental results on EMINST and CIFAR100 show that our method outperforms the recent LwF method, which use the knowledge distillation, and iCaRL method, which keep some old samples while training for new task. The method can achieve the goal of not forgetting old knowledge while training new classes, and solve the problem of catastrophic forgetting better.