Semi-Supervised Image-to-Image Translation using Latent Space Mapping

TL;DR

This paper proposes a semi-supervised image-to-image translation framework that operates in the latent space, improving translation quality and stability especially with limited paired data, outperforming existing unsupervised methods.

Contribution

It introduces a novel latent space mapping approach for semi-supervised image translation, enhancing results with minimal paired data compared to prior cycle-consistency based methods.

Findings

Better translation quality with limited paired data

Improved stability of the translation model

Outperforms state-of-the-art unsupervised methods

Abstract

Recent image-to-image translation works have been transferred from supervised to unsupervised settings due to the expensive cost of capturing or labeling large amounts of paired data. However, current unsupervised methods using the cycle-consistency constraint may not find the desired mapping, especially for difficult translation tasks. On the other hand, a small number of paired data are usually accessible. We therefore introduce a general framework for semi-supervised image translation. Unlike previous works, our main idea is to learn the translation over the latent feature space instead of the image space. Thanks to the low dimensional feature space, it is easier to find the desired mapping function, resulting in improved quality of translation results as well as the stability of the translation model. Empirically we show that using feature translation generates better results, even using a few bits of paired data. Experimental comparisons with state-of-the-art approaches demonstrate the effectiveness of the proposed framework on a variety of challenging image-to-image translation tasks