MotionV2V: Editing Motion in a Video
Ryan Burgert, Charles Herrmann, Forrester Cole, Michael S Ryoo, Neal Wadhwa, Andrey Voynov, and Nataniel Ruiz

TL;DR
MotionV2V introduces a novel method for editing existing videos by directly modifying sparse motion trajectories, enabling natural, flexible, and user-preferred video edits through a motion-conditioned diffusion model.
Contribution
The paper presents a new approach for video editing that uses sparse trajectory modifications and a motion-conditioned diffusion model, along with a dataset of motion counterfactuals for training.
Findings
Achieves over 65% user preference in comparisons
Enables edits starting at any timestamp with natural propagation
Demonstrates effective motion control in video editing
Abstract
While generative video models have achieved remarkable fidelity and consistency, applying these capabilities to video editing remains a complex challenge. Recent research has explored motion controllability as a means to enhance text-to-video generation or image animation; however, we identify precise motion control as a promising yet under-explored paradigm for editing existing videos. In this work, we propose modifying video motion by directly editing sparse trajectories extracted from the input. We term the deviation between input and output trajectories a "motion edit" and demonstrate that this representation, when coupled with a generative backbone, enables powerful video editing capabilities. To achieve this, we introduce a pipeline for generating "motion counterfactuals", video pairs that share identical content but distinct motion, and we fine-tune a motion-conditioned video…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Human Motion and Animation · Video Analysis and Summarization
