Tensor feature hallucination for few-shot learning

TL;DR

This paper introduces a simple tensor feature generation method for few-shot learning that outperforms complex existing data augmentation techniques, demonstrating the effectiveness of straightforward approaches.

Contribution

It proposes using a simple loss function and tensor features for synthetic data generation, achieving state-of-the-art results in few-shot classification.

Findings

Outperforms existing methods on miniImagenet, CUB, and CIFAR-FS datasets.

Simple loss functions suffice for effective feature generator training.

Tensor feature generation is superior to vector-based methods.

Abstract

Few-shot learning addresses the challenge of learning how to address novel tasks given not just limited supervision but limited data as well. An attractive solution is synthetic data generation. However, most such methods are overly sophisticated, focusing on high-quality, realistic data in the input space. It is unclear whether adapting them to the few-shot regime and using them for the downstream task of classification is the right approach. Previous works on synthetic data generation for few-shot classification focus on exploiting complex models, e.g. a Wasserstein GAN with multiple regularizers or a network that transfers latent diversities from known to novel classes. We follow a different approach and investigate how a simple and straightforward synthetic data generation method can be used effectively. We make two contributions, namely we show that: (1) using a simple loss function is more than enough for training a feature generator in the few-shot setting; and (2) learning to generate tensor features instead of vector features is superior. Extensive experiments on miniImagenet, CUB and CIFAR-FS datasets show that our method sets a new state of the art, outperforming more sophisticated few-shot data augmentation methods. The source code can be found at https://github.com/MichalisLazarou/TFH_fewshot.