Self-Denoising Neural Networks for Few Shot Learning

TL;DR

This paper introduces Self-Denoising Neural Networks (SDNNs), a new architecture that enhances few-shot learning by adding noise during training, leading to improved performance across multiple datasets and tasks.

Contribution

The paper proposes SDNN, a novel noise-robust training scheme for neural networks that boosts few-shot learning performance and can be integrated with existing architectures.

Findings

SDNN outperforms state-of-the-art methods on miniImageNet, tiered-ImageNet, and CIFAR-FS datasets.

Ablation studies validate the effectiveness of the SDNN architecture.

SDNN improves few-shot human action detection in videos.

Abstract

In this paper, we introduce a new architecture for few shot learning, the task of teaching a neural network from as few as one or five labeled examples. Inspired by the theoretical results of Alaine et al that Denoising Autoencoders refine features to lie closer to the true data manifold, we present a new training scheme that adds noise at multiple stages of an existing neural architecture while simultaneously learning to be robust to this added noise. This architecture, which we call a Self-Denoising Neural Network (SDNN), can be applied easily to most modern convolutional neural architectures, and can be used as a supplement to many existing few-shot learning techniques. We empirically show that SDNNs out-perform previous state-of-the-art methods for few shot image recognition using the Wide-ResNet architecture on the \textit{mini}ImageNet, tiered-ImageNet, and CIFAR-FS few shot learning datasets. We also perform a series of ablation experiments to empirically justify the construction of the SDNN architecture. Finally, we show that SDNNs even improve few shot performance on the task of human action detection in video using experiments on the ActEV SDL Surprise Activities challenge.