Zero-shot Image Editing with Reference Imitation
Xi Chen, Yutong Feng, Mengting Chen, Yiyang Wang, Shilong Zhang, Yu, Liu, Yujun Shen, Hengshuang Zhao

TL;DR
This paper introduces MimicBrush, a diffusion-based generative framework enabling zero-shot image editing by imitating reference images without requiring direct correspondence, thus enhancing user creativity and flexibility.
Contribution
The work presents a novel imitative editing paradigm and a self-supervised training method that captures semantic correspondence from in-the-wild references for flexible image editing.
Findings
Outperforms existing image editing methods in various scenarios
Effectively captures semantic relations between images without supervision
Establishes a new benchmark for reference-based image editing
Abstract
Image editing serves as a practical yet challenging task considering the diverse demands from users, where one of the hardest parts is to precisely describe how the edited image should look like. In this work, we present a new form of editing, termed imitative editing, to help users exercise their creativity more conveniently. Concretely, to edit an image region of interest, users are free to directly draw inspiration from some in-the-wild references (e.g., some relative pictures come across online), without having to cope with the fit between the reference and the source. Such a design requires the system to automatically figure out what to expect from the reference to perform the editing. For this purpose, we propose a generative training framework, dubbed MimicBrush, which randomly selects two frames from a video clip, masks some regions of one frame, and learns to recover the masked…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques · Advanced Vision and Imaging
MethodsDiffusion
