Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks

TL;DR

This paper introduces Covariance-Preserving Adversarial Augmentation Networks, a novel generative model that improves low-shot learning by modeling class-specific variability, leading to better data augmentation and performance on ImageNet.

Contribution

It proposes a new GAN architecture that explicitly preserves covariance information to generate diverse examples for low-shot learning.

Findings

Significant improvement over state-of-the-art on ImageNet

Generates realistic and diverse class examples

Effectively models class variability during augmentation

Abstract

Deep neural networks suffer from over-fitting and catastrophic forgetting when trained with small data. One natural remedy for this problem is data augmentation, which has been recently shown to be effective. However, previous works either assume that intra-class variances can always be generalized to new classes, or employ naive generation methods to hallucinate finite examples without modeling their latent distributions. In this work, we propose Covariance-Preserving Adversarial Augmentation Networks to overcome existing limits of low-shot learning. Specifically, a novel Generative Adversarial Network is designed to model the latent distribution of each novel class given its related base counterparts. Since direct estimation of novel classes can be inductively biased, we explicitly preserve covariance information as the `variability' of base examples during the generation process. Empirical results show that our model can generate realistic yet diverse examples, leading to substantial improvements on the ImageNet benchmark over the state of the art.