Can Synthetic Images Conquer Forgetting? Beyond Unexplored Doubts in Few-Shot Class-Incremental Learning

TL;DR

This paper introduces Diffusion-FSCIL, a novel method using a frozen diffusion model backbone for few-shot class-incremental learning, leveraging generative capabilities to reduce forgetting and improve adaptation.

Contribution

The paper proposes a new FSCIL approach employing a large pre-trained diffusion model with multi-scale features and minimal trainable components, enhancing continual learning performance.

Findings

Outperforms state-of-the-art on CUB-200, miniImageNet, and CIFAR-100.

Effectively preserves old class performance while learning new classes.

Utilizes a frozen backbone to improve efficiency and scalability.

Abstract

Few-shot class-incremental learning (FSCIL) is challenging due to extremely limited training data; while aiming to reduce catastrophic forgetting and learn new information. We propose Diffusion-FSCIL, a novel approach that employs a text-to-image diffusion model as a frozen backbone. Our conjecture is that FSCIL can be tackled using a large generative model's capabilities benefiting from 1) generation ability via large-scale pre-training; 2) multi-scale representation; 3) representational flexibility through the text encoder. To maximize the representation capability, we propose to extract multiple complementary diffusion features to play roles as latent replay with slight support from feature distillation for preventing generative biases. Our framework realizes efficiency through 1) using a frozen backbone; 2) minimal trainable components; 3) batch processing of multiple feature extractions. Extensive experiments on CUB-200, \emph{mini}ImageNet, and CIFAR-100 show that Diffusion-FSCIL surpasses state-of-the-art methods, preserving performance on previously learned classes and adapting effectively to new ones.