Class-Incremental Exemplar Compression for Class-Incremental Learning

TL;DR

This paper introduces a novel exemplar compression method for class-incremental learning that uses adaptive masks generated from class activation maps to improve accuracy without manual annotation.

Contribution

It proposes the class-incremental masking (CIM) model that adaptively generates discriminative pixel masks for exemplar compression in CIL, surpassing existing methods.

Findings

Achieves state-of-the-art accuracy on high-resolution CIL benchmarks.

Improves ImageNet-1000 accuracy by 4.8 percentage points over FOSTER.

Effectively balances exemplar coverage and quantity through adaptive masking.

Abstract

Exemplar-based class-incremental learning (CIL) finetunes the model with all samples of new classes but few-shot exemplars of old classes in each incremental phase, where the "few-shot" abides by the limited memory budget. In this paper, we break this "few-shot" limit based on a simple yet surprisingly effective idea: compressing exemplars by downsampling non-discriminative pixels and saving "many-shot" compressed exemplars in the memory. Without needing any manual annotation, we achieve this compression by generating 0-1 masks on discriminative pixels from class activation maps (CAM). We propose an adaptive mask generation model called class-incremental masking (CIM) to explicitly resolve two difficulties of using CAM: 1) transforming the heatmaps of CAM to 0-1 masks with an arbitrary threshold leads to a trade-off between the coverage on discriminative pixels and the quantity of exemplars, as the total memory is fixed; and 2) optimal thresholds vary for different object classes, which is particularly obvious in the dynamic environment of CIL. We optimize the CIM model alternatively with the conventional CIL model through a bilevel optimization problem. We conduct extensive experiments on high-resolution CIL benchmarks including Food-101, ImageNet-100, and ImageNet-1000, and show that using the compressed exemplars by CIM can achieve a new state-of-the-art CIL accuracy, e.g., 4.8 percentage points higher than FOSTER on 10-Phase ImageNet-1000. Our code is available at https://github.com/xfflzl/CIM-CIL.