BrushEdit: All-In-One Image Inpainting and Editing
Yaowei Li, Yuxuan Bian, Xuan Ju, Zhaoyang Zhang, Junhao Zhuang, Ying, Shan, Yuexian Zou, Qiang Xu

TL;DR
BrushEdit introduces a novel image editing framework that combines multimodal large language models and inpainting techniques to enable autonomous, interactive, and precise image modifications guided by free-form instructions.
Contribution
The paper presents a new inpainting-based instruction-guided editing paradigm that overcomes limitations of existing inversion and instruction-based methods by integrating MLLMs with inpainting models in an agent-cooperative system.
Findings
Effective in performing large modifications like object addition/removal.
Achieves superior performance on multiple metrics including mask preservation and editing coherence.
Enables user-friendly, interactive image editing with free-form instructions.
Abstract
Image editing has advanced significantly with the development of diffusion models using both inversion-based and instruction-based methods. However, current inversion-based approaches struggle with big modifications (e.g., adding or removing objects) due to the structured nature of inversion noise, which hinders substantial changes. Meanwhile, instruction-based methods often constrain users to black-box operations, limiting direct interaction for specifying editing regions and intensity. To address these limitations, we propose BrushEdit, a novel inpainting-based instruction-guided image editing paradigm, which leverages multimodal large language models (MLLMs) and image inpainting models to enable autonomous, user-friendly, and interactive free-form instruction editing. Specifically, we devise a system enabling free-form instruction editing by integrating MLLMs and a dual-branch image…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis
MethodsDiffusion · Inpainting
