Few Shot Class Incremental Learning using Vision-Language models

TL;DR

This paper introduces a novel few-shot class incremental learning framework leveraging vision-language models with regularizers to incorporate semantic information and maintain performance on base classes, achieving state-of-the-art results.

Contribution

The proposed framework uniquely combines language and subspace regularizers to improve incremental learning with limited data for new classes.

Findings

Achieves state-of-the-art performance on three FSCIL benchmarks.

Effectively incorporates semantic information from vision-language models.

Maintains performance on base classes during incremental learning.

Abstract

Recent advancements in deep learning have demonstrated remarkable performance comparable to human capabilities across various supervised computer vision tasks. However, the prevalent assumption of having an extensive pool of training data encompassing all classes prior to model training often diverges from real-world scenarios, where limited data availability for novel classes is the norm. The challenge emerges in seamlessly integrating new classes with few samples into the training data, demanding the model to adeptly accommodate these additions without compromising its performance on base classes. To address this exigency, the research community has introduced several solutions under the realm of few-shot class incremental learning (FSCIL). In this study, we introduce an innovative FSCIL framework that utilizes language regularizer and subspace regularizer. During base training, the language regularizer helps incorporate semantic information extracted from a Vision-Language model. The subspace regularizer helps in facilitating the model's acquisition of nuanced connections between image and text semantics inherent to base classes during incremental training. Our proposed framework not only empowers the model to embrace novel classes with limited data, but also ensures the preservation of performance on base classes. To substantiate the efficacy of our approach, we conduct comprehensive experiments on three distinct FSCIL benchmarks, where our framework attains state-of-the-art performance.