Rotation Augmented Distillation for Exemplar-Free Class Incremental Learning with Detailed Analysis

TL;DR

This paper introduces Rotation Augmented Distillation (RAD), a novel method for exemplar-free class incremental learning that effectively balances plasticity and stability, achieving top-tier performance without using old class samples.

Contribution

The paper proposes RAD, a simple yet effective exemplar-free CIL method, and provides detailed analysis and evaluation in more challenging settings.

Findings

RAD achieves top-tier performance in exemplar-free CIL.

RAD maintains a better balance between plasticity and stability.

The detailed analysis offers insights into the method's effectiveness.

Abstract

Class incremental learning (CIL) aims to recognize both the old and new classes along the increment tasks. Deep neural networks in CIL suffer from catastrophic forgetting and some approaches rely on saving exemplars from previous tasks, known as the exemplar-based setting, to alleviate this problem. On the contrary, this paper focuses on the Exemplar-Free setting with no old class sample preserved. Balancing the plasticity and stability in deep feature learning with only supervision from new classes is more challenging. Most existing Exemplar-Free CIL methods report the overall performance only and lack further analysis. In this work, different methods are examined with complementary metrics in greater detail. Moreover, we propose a simple CIL method, Rotation Augmented Distillation (RAD), which achieves one of the top-tier performances under the Exemplar-Free setting. Detailed analysis shows our RAD benefits from the superior balance between plasticity and stability. Finally, more challenging exemplar-free settings with fewer initial classes are undertaken for further demonstrations and comparisons among the state-of-the-art methods.