Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning

TL;DR

This paper introduces a neural collapse inspired framework for few-shot class-incremental learning that maintains feature-classifier alignment and outperforms existing methods on multiple datasets.

Contribution

It proposes a novel approach using fixed simplex ETF classifier prototypes and a specialized loss to prevent misalignment and catastrophic forgetting in FSCIL.

Findings

Outperforms state-of-the-art on miniImageNet, CUB-200, CIFAR-100

Maintains feature-classifier alignment during incremental learning

Theoretically guarantees neural collapse optimality

Abstract

Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. Finetuning the backbone or adjusting the classifier prototypes trained in the prior sessions would inevitably cause a misalignment between the feature and classifier of old classes, which explains the well-known catastrophic forgetting problem. In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF). It corresponds to an optimal geometric structure for classification due to the maximized Fisher Discriminant Ratio. We propose a neural collapse inspired framework for FSCIL. A group of classifier prototypes are pre-assigned as a simplex ETF for the whole label space, including the base session and all the incremental sessions. During training, the classifier prototypes are not learnable, and we adopt a novel loss function that drives the features into their corresponding prototypes. Theoretical analysis shows that our method holds the neural collapse optimality and does not break the feature-classifier alignment in an incremental fashion. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances. Code address: https://github.com/NeuralCollapseApplications/FSCIL