Few-Shot Lifelong Learning

TL;DR

This paper introduces a novel Few-Shot Lifelong Learning method that selects a small subset of model parameters for training new classes, effectively preventing overfitting and catastrophic forgetting, and significantly improving classification performance on multiple datasets.

Contribution

The proposed FSLL method uniquely selects unimportant parameters for training new classes, combining parameter selection with prototype separation and self-supervision to enhance lifelong learning in few-shot scenarios.

Findings

Outperforms existing methods on miniImageNet, CIFAR-100, and CUB-200 datasets.

Achieves a 19.27% accuracy improvement on the CUB dataset over state-of-the-art.

Effectively prevents catastrophic forgetting while improving class separation.

Abstract

Many real-world classification problems often have classes with very few labeled training samples. Moreover, all possible classes may not be initially available for training, and may be given incrementally. Deep learning models need to deal with this two-fold problem in order to perform well in real-life situations. In this paper, we propose a novel Few-Shot Lifelong Learning (FSLL) method that enables deep learning models to perform lifelong/continual learning on few-shot data. Our method selects very few parameters from the model for training every new set of classes instead of training the full model. This helps in preventing overfitting. We choose the few parameters from the model in such a way that only the currently unimportant parameters get selected. By keeping the important parameters in the model intact, our approach minimizes catastrophic forgetting. Furthermore, we minimize the cosine similarity between the new and the old class prototypes in order to maximize their separation, thereby improving the classification performance. We also show that integrating our method with self-supervision improves the model performance significantly. We experimentally show that our method significantly outperforms existing methods on the miniImageNet, CIFAR-100, and CUB-200 datasets. Specifically, we outperform the state-of-the-art method by an absolute margin of 19.27% for the CUB dataset.