Diffusion-Based Image-to-Image Translation by Noise Correction via Prompt Interpolation

TL;DR

This paper introduces a training-free, noise correction method for diffusion-based image-to-image translation that improves editing precision by interpolating prompt embeddings, enhancing existing models' performance.

Contribution

It presents a novel noise correction technique using prompt interpolation, enabling effective, low-latency image translation without retraining diffusion models.

Findings

Achieves superior translation quality compared to baseline methods.

Enhances existing diffusion-based frameworks when integrated.

Maintains low latency during image translation.

Abstract

We propose a simple but effective training-free approach tailored to diffusion-based image-to-image translation. Our approach revises the original noise prediction network of a pretrained diffusion model by introducing a noise correction term. We formulate the noise correction term as the difference between two noise predictions; one is computed from the denoising network with a progressive interpolation of the source and target prompt embeddings, while the other is the noise prediction with the source prompt embedding. The final noise prediction network is given by a linear combination of the standard denoising term and the noise correction term, where the former is designed to reconstruct must-be-preserved regions while the latter aims to effectively edit regions of interest relevant to the target prompt. Our approach can be easily incorporated into existing image-to-image translation methods based on diffusion models. Extensive experiments verify that the proposed technique achieves outstanding performance with low latency and consistently improves existing frameworks when combined with them.