Delta Denoising Score

TL;DR

Delta Denoising Score (DDS) is a new scoring function that improves text-based image editing by guiding minimal modifications using diffusion models, addressing noise issues in existing methods and enabling effective zero-shot translation.

Contribution

The paper introduces DDS, a novel scoring function that enhances image editing stability and quality by refining score distillation sampling with prompt matching.

Findings

DDS outperforms existing methods in stability and quality

DDS enables effective zero-shot image translation

DDS reduces noise and blurriness in edited images

Abstract

We introduce Delta Denoising Score (DDS), a novel scoring function for text-based image editing that guides minimal modifications of an input image towards the content described in a target prompt. DDS leverages the rich generative prior of text-to-image diffusion models and can be used as a loss term in an optimization problem to steer an image towards a desired direction dictated by a text. DDS utilizes the Score Distillation Sampling (SDS) mechanism for the purpose of image editing. We show that using only SDS often produces non-detailed and blurry outputs due to noisy gradients. To address this issue, DDS uses a prompt that matches the input image to identify and remove undesired erroneous directions of SDS. Our key premise is that SDS should be zero when calculated on pairs of matched prompts and images, meaning that if the score is non-zero, its gradients can be attributed to the erroneous component of SDS. Our analysis demonstrates the competence of DDS for text based image-to-image translation. We further show that DDS can be used to train an effective zero-shot image translation model. Experimental results indicate that DDS outperforms existing methods in terms of stability and quality, highlighting its potential for real-world applications in text-based image editing.