Few-shot learning using pre-training and shots, enriched by pre-trained samples

TL;DR

This paper presents a simple few-shot learning method using pre-trained neural networks and a novel approach of fixing the first layer during training, achieving high accuracy with minimal samples.

Contribution

It introduces a new few-shot learning technique that leverages pre-trained models and a specific training protocol to improve accuracy with few examples.

Findings

Achieved about 90% accuracy after 10 shots.

Method effectively utilizes pre-trained samples and fixed first layer.

Simple approach outperforms some existing few-shot methods.

Abstract

We use the EMNIST dataset of handwritten digits to test a simple approach for few-shot learning. A fully connected neural network is pre-trained with a subset of the 10 digits and used for few-shot learning with untrained digits. Two basic ideas are introduced: during few-shot learning the learning of the first layer is disabled, and for every shot a previously unknown digit is used together with four previously trained digits for the gradient descend, until a predefined threshold condition is fulfilled. This way we reach about 90% accuracy after 10 shots.