EditWorld: Simulating World Dynamics for Instruction-Following Image Editing
Ling Yang, Bohan Zeng, Jiaming Liu, Hong Li, Minghao Xu, Wentao Zhang,, Shuicheng Yan

TL;DR
EditWorld introduces a novel instruction-based image editing task focused on simulating world dynamics, utilizing a curated dataset and training strategies to enhance realistic, dynamic scene editing beyond simple modifications.
Contribution
The paper presents a new world-instructed image editing task, a curated dataset with world instructions, and a training method that improves instruction-following for dynamic scene editing.
Findings
Outperforms existing editing methods in the new task
Demonstrates effective simulation of world dynamics in image editing
Provides a dataset and code for further research
Abstract
Diffusion models have significantly improved the performance of image editing. Existing methods realize various approaches to achieve high-quality image editing, including but not limited to text control, dragging operation, and mask-and-inpainting. Among these, instruction-based editing stands out for its convenience and effectiveness in following human instructions across diverse scenarios. However, it still focuses on simple editing operations like adding, replacing, or deleting, and falls short of understanding aspects of world dynamics that convey the realistic dynamic nature in the physical world. Therefore, this work, EditWorld, introduces a new editing task, namely world-instructed image editing, which defines and categorizes the instructions grounded by various world scenarios. We curate a new image editing dataset with world instructions using a set of large pretrained models…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemantic Web and Ontologies · Scientific Computing and Data Management · Distributed and Parallel Computing Systems
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · 15 Ways to Contact How can i speak to someone at Delta Airlines · Attention Is All You Need · Sparse Evolutionary Training · Linear Layer · Residual Connection · Byte Pair Encoding · Adam · Dropout · Softmax
