Replace Anyone in Videos
Xiang Wang, Shiwei Zhang, Haonan Qiu, Ruihang Chu, Zekun Li, Yingya, Zhang, Changxin Gao, Yuehuan Wang, Chunhua Shen, Nong Sang

TL;DR
ReplaceAnyone introduces a novel end-to-end diffusion-based framework for localized human replacement and insertion in videos, enabling precise control over pose and appearance while preserving backgrounds and motion coherence.
Contribution
The paper presents a unified video diffusion architecture with mask diversity, enhanced guidance, and a two-phase training approach for realistic human-centric video editing.
Findings
Effective in realistic human replacement and insertion
Preserves background details and motion coherence
Compatible with multiple diffusion model architectures
Abstract
The field of controllable human-centric video generation has witnessed remarkable progress, particularly with the advent of diffusion models. However, achieving precise and localized control over human motion in videos, such as replacing or inserting individuals while preserving desired motion patterns, still remains a formidable challenge. In this work, we present the ReplaceAnyone framework, which focuses on localized human replacement and insertion featuring intricate backgrounds. Specifically, we formulate this task as an image-conditioned video inpainting paradigm with pose guidance, utilizing a unified end-to-end video diffusion architecture that facilitates image-conditioned video inpainting within masked regions. To prevent shape leakage and enable granular local control, we introduce diverse mask forms involving both regular and irregular shapes. Furthermore, we implement an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCinema and Media Studies · Digital Games and Media
MethodsBalanced Selection · Inpainting · Diffusion
