CannyEdit: Selective Canny Control and Dual-Prompt Guidance for Training-Free Image Editing

TL;DR

CannyEdit is a training-free image editing framework that uses selective structural guidance and dual-prompt strategies to improve control, fidelity, and seamlessness in region-specific edits, even with minimal input.

Contribution

It introduces a novel training-free approach combining Selective Canny Control and Dual-Prompt Guidance for improved image editing control and flexibility.

Findings

Outperforms existing region-based editing methods in text adherence and seamlessness.

Effective with rough masks or single-point hints for additional tasks.

Seamlessly integrates with vision-language models for complex instruction-based editing.

Abstract

Recent advances in text-to-image (T2I) models have enabled training-free regional image editing by leveraging the generative priors of foundation models. However, existing methods struggle to balance text adherence in edited regions, context fidelity in unedited areas, and seamless integration of edits. We introduce CannyEdit, a novel training-free framework that addresses this trilemma through two key innovations. First, Selective Canny Control applies structural guidance from a Canny ControlNet only to the unedited regions, preserving the original image's details while allowing for precise, text-driven changes in the specified editable area. Second, Dual-Prompt Guidance utilizes both a local prompt for the specific edit and a global prompt for overall scene coherence. Through this synergistic approach, these components enable controllable local editing for object addition, replacement, and removal, achieving a superior trade-off among text adherence, context fidelity, and editing seamlessness compared to current region-based methods. Beyond this, CannyEdit offers exceptional flexibility: it operates effectively with rough masks or even single-point hints in addition tasks. Furthermore, the framework can seamlessly integrate with vision-language models in a training-free manner for complex instruction-based editing that requires planning and reasoning. Our extensive evaluations demonstrate CannyEdit's strong performance against leading instruction-based editors in complex object addition scenarios.