Toward Zero-Shot Unsupervised Image-to-Image Translation

TL;DR

This paper introduces a zero-shot unsupervised image-to-image translation framework that leverages semantic attributes to generalize to unseen classes, overcoming mode collapse when target data is scarce.

Contribution

It proposes a novel method that uses semantic attribute space to enable zero-shot translation, addressing limitations of existing unsupervised methods.

Findings

Effective in translating images to unseen classes

Improves mode exploration in target space

Applicable to zero-shot classification and fashion design

Abstract

Recent studies have shown remarkable success in unsupervised image-to-image translation. However, if there has no access to enough images in target classes, learning a mapping from source classes to the target classes always suffers from mode collapse, which limits the application of the existing methods. In this work, we propose a zero-shot unsupervised image-to-image translation framework to address this limitation, by associating categories with their side information like attributes. To generalize the translator to previous unseen classes, we introduce two strategies for exploiting the space spanned by the semantic attributes. Specifically, we propose to preserve semantic relations to the visual space and expand attribute space by utilizing attribute vectors of unseen classes, thus encourage the translator to explore the modes of unseen classes. Quantitative and qualitative results on different datasets demonstrate the effectiveness of our proposed approach. Moreover, we demonstrate that our framework can be applied to many tasks, such as zero-shot classification and fashion design.