One-Shot Learning in Discriminative Neural Networks

TL;DR

This paper introduces a Bayesian approach for one-shot learning in neural networks, enabling effective classification of new categories with limited data by updating a pretrained convnet using a Gaussian prior.

Contribution

It proposes a Bayesian method that updates a pretrained convnet for one-shot learning, combining a fixed feature extractor with a Gaussian prior for new class weights.

Findings

Achieves competitive performance with state-of-the-art methods

Maintains consistency with traditional deep learning training on large datasets

Provides a probabilistic framework for one-shot learning

Abstract

We consider the task of one-shot learning of visual categories. In this paper we explore a Bayesian procedure for updating a pretrained convnet to classify a novel image category for which data is limited. We decompose this convnet into a fixed feature extractor and softmax classifier. We assume that the target weights for the new task come from the same distribution as the pretrained softmax weights, which we model as a multivariate Gaussian. By using this as a prior for the new weights, we demonstrate competitive performance with state-of-the-art methods whilst also being consistent with 'normal' methods for training deep networks on large data.