Rethinking Few-shot Class-incremental Learning: Learning from Yourself

TL;DR

This paper introduces a new evaluation metric called generalized average accuracy (gAcc) for FSCIL, and leverages intermediate transformer features to improve novel class performance without complex training, outperforming existing methods.

Contribution

It proposes a novel evaluation metric gAcc and a transformer-based framework utilizing intermediate features to enhance FSCIL performance.

Findings

gAcc provides a more balanced evaluation of class performance.

Using intermediate transformer features boosts novel class accuracy.

The method outperforms existing FSCIL approaches on multiple datasets.

Abstract

Few-shot class-incremental learning (FSCIL) aims to learn sequential classes with limited samples in a few-shot fashion. Inherited from the classical class-incremental learning setting, the popular benchmark of FSCIL uses averaged accuracy (aAcc) and last-task averaged accuracy (lAcc) as the evaluation metrics. However, we reveal that such evaluation metrics may not provide adequate emphasis on the novel class performance, and the continual learning ability of FSCIL methods could be ignored under this benchmark. In this work, as a complement to existing metrics, we offer a new metric called generalized average accuracy (gAcc) which is designed to provide an extra equitable evaluation by incorporating different perspectives of the performance under the guidance of a parameter $\alpha$. We also present an overall metric in the form of the area under the curve (AUC) along the $\alpha$. Under the guidance of gAcc, we release the potential of intermediate features of the vision transformers to boost the novel-class performance. Taking information from intermediate layers which are less class-specific and more generalizable, we manage to rectify the final features, leading to a more generalizable transformer-based FSCIL framework. Without complex network designs or cumbersome training procedures, our method outperforms existing FSCIL methods at aAcc and gAcc on three datasets. See codes at https://github.com/iSEE-Laboratory/Revisting_FSCIL