ReVideo: Remake a Video with Motion and Content Control
Chong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang, Ying Shan, Jian, Zhang

TL;DR
ReVideo introduces a novel video editing framework that enables precise, localized content and motion modifications using a three-stage training strategy and a spatiotemporal fusion module, advancing the accuracy and flexibility of diffusion-based video editing.
Contribution
The paper presents a new method for coupled content and motion editing in videos, with a three-stage training process and a fusion module, addressing coupling and training imbalance issues.
Findings
Effective local content modification while maintaining motion.
Ability to customize motion trajectories without altering content.
Supports multi-area editing without additional training.
Abstract
Despite significant advancements in video generation and editing using diffusion models, achieving accurate and localized video editing remains a substantial challenge. Additionally, most existing video editing methods primarily focus on altering visual content, with limited research dedicated to motion editing. In this paper, we present a novel attempt to Remake a Video (ReVideo) which stands out from existing methods by allowing precise video editing in specific areas through the specification of both content and motion. Content editing is facilitated by modifying the first frame, while the trajectory-based motion control offers an intuitive user interaction experience. ReVideo addresses a new task involving the coupling and training imbalance between content and motion control. To tackle this, we develop a three-stage training strategy that progressively decouples these two aspects…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsMultimedia Communication and Technology · Cinema and Media Studies
MethodsFocus · Diffusion
