Feature-Space Generative Models for One-Shot Class-Incremental Learning

TL;DR

This paper introduces Gen1S, a generative modeling approach that enhances one-shot class-incremental learning by leveraging residual space and structural priors, significantly improving recognition of novel classes.

Contribution

The paper proposes a novel residual space mapping and generative modeling framework for 1-shot class-incremental learning, addressing the challenge of recognizing novel classes without further training.

Findings

Gen1S outperforms state-of-the-art methods across multiple benchmarks.

Using generative models of residuals improves recognition of novel classes.

Structural priors based on residuals enhance generalization to new classes.

Abstract

Few-shot class-incremental learning (FSCIL) is a paradigm where a model, initially trained on a dataset of base classes, must adapt to an expanding problem space by recognizing novel classes with limited data. We focus on the challenging FSCIL setup where a model receives only a single sample (1-shot) for each novel class and no further training or model alterations are allowed after the base training phase. This makes generalization to novel classes particularly difficult. We propose a novel approach predicated on the hypothesis that base and novel class embeddings have structural similarity. We map the original embedding space into a residual space by subtracting the class prototype (i.e., the average class embedding) of input samples. Then, we leverage generative modeling with VAE or diffusion models to learn the multi-modal distribution of residuals over the base classes, and we use this as a valuable structural prior to improve recognition of novel classes. Our approach, Gen1S, consistently improves novel class recognition over the state of the art across multiple benchmarks and backbone architectures.