Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps

TL;DR

This paper introduces invertible Consistency Distillation (iCD), a framework enabling high-quality, fast encoding and editing of real images in text-guided diffusion models within 3-4 steps, enhancing capabilities for image manipulation.

Contribution

The work presents a novel invertible consistency distillation method that improves real image inversion and editing in text-to-image diffusion models with minimal inference steps.

Findings

iCD achieves high-quality image synthesis and encoding in 3-4 steps

Dynamic guidance reduces reconstruction errors during inversion

iCD enables effective zero-shot text-guided image editing

Abstract

Diffusion distillation represents a highly promising direction for achieving faithful text-to-image generation in a few sampling steps. However, despite recent successes, existing distilled models still do not provide the full spectrum of diffusion abilities, such as real image inversion, which enables many precise image manipulation methods. This work aims to enrich distilled text-to-image diffusion models with the ability to effectively encode real images into their latent space. To this end, we introduce invertible Consistency Distillation (iCD), a generalized consistency distillation framework that facilitates both high-quality image synthesis and accurate image encoding in only 3-4 inference steps. Though the inversion problem for text-to-image diffusion models gets exacerbated by high classifier-free guidance scales, we notice that dynamic guidance significantly reduces reconstruction errors without noticeable degradation in generation performance. As a result, we demonstrate that iCD equipped with dynamic guidance may serve as a highly effective tool for zero-shot text-guided image editing, competing with more expensive state-of-the-art alternatives.