# Expanding continual few-shot learning benchmarks to include recognition of specific instances

## TL;DR

This paper expands a machine learning benchmark to better simulate real-world challenges by increasing class numbers and testing instance recognition.

## Contribution

The paper introduces a new benchmark with more classes and an instance recognition test to evaluate continual few-shot learning.

## Key findings

- Learning more classes is more challenging than previous continual few-shot learning experiments.
- Instance recognition performance is comparable to other tasks but drops significantly with occlusion and noise.
- Using replay improves performance, especially for instance recognition tasks.

## Abstract

Continual learning and few-shot learning are important frontiers in progress toward broader Machine Learning (ML) capabilities. Recently, there has been intense interest in combining both. One of the first examples to do so was the Continual few-shot Learning (CFSL) framework of Antoniou et al. (2020). In this study, we extend CFSL in two ways that capture a broader range of challenges, important for intelligent agent behaviour in real-world conditions. First, we increased the number of classes by an order of magnitude, making the results more comparable to standard continual learning experiments. Second, we introduced an ‘instance test’ which requires recognition of specific instances of classes—a capability of animal cognition that is usually neglected in ML. For an initial exploration of ML model performance under these conditions, we selected representative baseline models from the original CFSL work and added a model variant with replay. As expected, learning more classes is more difficult than the original CFSL experiments, and interestingly, the way in which image instances and classes are presented affects classification performance. Surprisingly, accuracy in the baseline instance test is comparable to other classification tasks, but poor given significant occlusion and noise. The use of replay for consolidation substantially improves performance for both types of tasks, but particularly for the instance test.