Coarse-To-Fine Incremental Few-Shot Learning

TL;DR

This paper introduces a new approach called Knowe for coarse-to-fine incremental few-shot learning, effectively recognizing fine-grained classes over time without forgetting previous classes, and outperforms existing methods on multiple datasets.

Contribution

It formulates the hybrid coarse-to-fine recognition as a class-incremental learning problem and proposes a simple, theoretically-sound strategy to learn and freeze classifier weights from coarse labels.

Findings

Knowe outperforms recent CIL/FSCIL methods on CIFAR-100, BREEDS, and tieredImageNet.

The proposed method effectively balances stability and plasticity in incremental learning.

New performance metrics are introduced for better evaluation of CIL in coarse-to-fine scenarios.

Abstract

Different from fine-tuning models pre-trained on a large-scale dataset of preset classes, class-incremental learning (CIL) aims to recognize novel classes over time without forgetting pre-trained classes. However, a given model will be challenged by test images with finer-grained classes, e.g., a basenji is at most recognized as a dog. Such images form a new training set (i.e., support set) so that the incremental model is hoped to recognize a basenji (i.e., query) as a basenji next time. This paper formulates such a hybrid natural problem of coarse-to-fine few-shot (C2FS) recognition as a CIL problem named C2FSCIL, and proposes a simple, effective, and theoretically-sound strategy Knowe: to learn, normalize, and freeze a classifier's weights from fine labels, once learning an embedding space contrastively from coarse labels. Besides, as CIL aims at a stability-plasticity balance, new overall performance metrics are proposed. In that sense, on CIFAR-100, BREEDS, and tieredImageNet, Knowe outperforms all recent relevant CIL/FSCIL methods that are tailored to the new problem setting for the first time.