Zero-shot Learning via Simultaneous Generating and Learning

TL;DR

This paper introduces a deep generative model for zero-shot learning that learns to generate data for unseen classes by treating missing data as a learning problem, enabling direct classification and generation.

Contribution

It proposes a variational auto-encoder with class-specific multi-modal prior to model seen and unseen classes simultaneously, improving zero-shot learning performance.

Findings

Achieves state-of-the-art results on zero-shot learning benchmarks.

Outperforms models trained only on seen classes.

Enables direct classification and data generation for unseen classes.

Abstract

To overcome the absence of training data for unseen classes, conventional zero-shot learning approaches mainly train their model on seen datapoints and leverage the semantic descriptions for both seen and unseen classes. Beyond exploiting relations between classes of seen and unseen, we present a deep generative model to provide the model with experience about both seen and unseen classes. Based on the variational auto-encoder with class-specific multi-modal prior, the proposed method learns the conditional distribution of seen and unseen classes. In order to circumvent the need for samples of unseen classes, we treat the non-existing data as missing examples. That is, our network aims to find optimal unseen datapoints and model parameters, by iteratively following the generating and learning strategy. Since we obtain the conditional generative model for both seen and unseen classes, classification as well as generation can be performed directly without any off-the-shell classifiers. In experimental results, we demonstrate that the proposed generating and learning strategy makes the model achieve the outperforming results compared to that trained only on the seen classes, and also to the several state-of-the-art methods.