Language-Free Generative Editing from One Visual Example

TL;DR

This paper introduces Visual Diffusion Conditioning (VDC), a training-free, vision-centric approach for precise image editing using visual examples, overcoming limitations of text-based diffusion models in simple transformations.

Contribution

VDC is a novel, training-free framework that derives visual conditions directly from examples for accurate, language-free image editing, improving over existing text-based methods.

Findings

VDC outperforms text-based editing methods across diverse tasks.

VDC preserves fine details and realism through inversion-correction.

VDC requires no additional training or fine-tuning.

Abstract

Text-guided diffusion models have advanced image editing by enabling intuitive control through language. However, despite their strong capabilities, we surprisingly find that SOTA methods struggle with simple, everyday transformations such as rain or blur. We attribute this limitation to weak and inconsistent textual supervision during training, which leads to poor alignment between language and vision. Existing solutions often rely on extra finetuning or stronger text conditioning, but suffer from high data and computational requirements. We argue that diffusion-based editing capabilities aren't lost but merely hidden from text. The door to cost-efficient visual editing remains open, and the key lies in a vision-centric paradigm that perceives and reasons about visual change as humans do, beyond words. Inspired by this, we introduce Visual Diffusion Conditioning (VDC), a training-free framework that learns conditioning signals directly from visual examples for precise, language-free image editing. Given a paired example -one image with and one without the target effect- VDC derives a visual condition that captures the transformation and steers generation through a novel condition-steering mechanism. An accompanying inversion-correction step mitigates reconstruction errors during DDIM inversion, preserving fine detail and realism. Across diverse tasks, VDC outperforms both training-free and fully fine-tuned text-based editing methods. The code and models are open-sourced at https://omaralezaby.github.io/vdc/