Domain-Scalable Unpaired Image Translation via Latent Space Anchoring

TL;DR

This paper introduces a scalable unpaired image translation method that efficiently incorporates new domains without retraining the entire model, using latent space anchoring with lightweight encoders and decoders.

Contribution

The proposed latent space anchoring approach enables domain-scalable image translation by avoiding full retraining and fine-tuning, allowing flexible combination of encoders and decoders for new domains.

Findings

Achieves superior performance on standard and scalable UNIT tasks.

Does not require fine-tuning of existing encoders and decoders.

Efficiently extends to new visual domains with lightweight models.

Abstract

Unpaired image-to-image translation (UNIT) aims to map images between two visual domains without paired training data. However, given a UNIT model trained on certain domains, it is difficult for current methods to incorporate new domains because they often need to train the full model on both existing and new domains. To address this problem, we propose a new domain-scalable UNIT method, termed as latent space anchoring, which can be efficiently extended to new visual domains and does not need to fine-tune encoders and decoders of existing domains. Our method anchors images of different domains to the same latent space of frozen GANs by learning lightweight encoder and regressor models to reconstruct single-domain images. In the inference phase, the learned encoders and decoders of different domains can be arbitrarily combined to translate images between any two domains without fine-tuning. Experiments on various datasets show that the proposed method achieves superior performance on both standard and domain-scalable UNIT tasks in comparison with the state-of-the-art methods.