General Image-to-Image Translation with One-Shot Image Guidance

TL;DR

This paper introduces a novel framework called visual concept translator (VCT) that enables content-preserving, one-shot image-to-image translation guided by a single reference image, outperforming existing methods in flexibility and quality.

Contribution

The paper presents VCT, a new framework with content-concept inversion and fusion processes for effective one-shot image translation guided by a single reference image.

Findings

VCT effectively preserves source image content while translating visual concepts.

VCT achieves superior results across various image translation tasks.

Extensive experiments demonstrate the method's effectiveness and superiority.

Abstract

Large-scale text-to-image models pre-trained on massive text-image pairs show excellent performance in image synthesis recently. However, image can provide more intuitive visual concepts than plain text. People may ask: how can we integrate the desired visual concept into an existing image, such as our portrait? Current methods are inadequate in meeting this demand as they lack the ability to preserve content or translate visual concepts effectively. Inspired by this, we propose a novel framework named visual concept translator (VCT) with the ability to preserve content in the source image and translate the visual concepts guided by a single reference image. The proposed VCT contains a content-concept inversion (CCI) process to extract contents and concepts, and a content-concept fusion (CCF) process to gather the extracted information to obtain the target image. Given only one reference image, the proposed VCT can complete a wide range of general image-to-image translation tasks with excellent results. Extensive experiments are conducted to prove the superiority and effectiveness of the proposed methods. Codes are available at https://github.com/CrystalNeuro/visual-concept-translator.