EBDM: Exemplar-guided Image Translation with Brownian-bridge Diffusion Models

TL;DR

This paper introduces EBDM, a novel exemplar-guided image translation method using Brownian-bridge diffusion models, which efficiently incorporates style from exemplars without dense correspondence, improving robustness and performance.

Contribution

The paper proposes a new diffusion-based framework for exemplar-guided image translation that avoids dense correspondence, reducing computational costs and enhancing versatility.

Findings

Outperforms existing methods in benchmark evaluations.

Effectively captures style from exemplars with global and detailed texture integration.

Achieves robust translation with efficient training and inference.

Abstract

Exemplar-guided image translation, synthesizing photo-realistic images that conform to both structural control and style exemplars, is attracting attention due to its ability to enhance user control over style manipulation. Previous methodologies have predominantly depended on establishing dense correspondences across cross-domain inputs. Despite these efforts, they incur quadratic memory and computational costs for establishing dense correspondence, resulting in limited versatility and performance degradation. In this paper, we propose a novel approach termed Exemplar-guided Image Translation with Brownian-Bridge Diffusion Models (EBDM). Our method formulates the task as a stochastic Brownian bridge process, a diffusion process with a fixed initial point as structure control and translates into the corresponding photo-realistic image while being conditioned solely on the given exemplar image. To efficiently guide the diffusion process toward the style of exemplar, we delineate three pivotal components: the Global Encoder, the Exemplar Network, and the Exemplar Attention Module to incorporate global and detailed texture information from exemplar images. Leveraging Bridge diffusion, the network can translate images from structure control while exclusively conditioned on the exemplar style, leading to more robust training and inference processes. We illustrate the superiority of our method over competing approaches through comprehensive benchmark evaluations and visual results.