TrajectoryMover: Generative Movement of Object Trajectories in Videos

TL;DR

TrajectoryMover is a novel method that enables the generative movement of object trajectories in videos by using a new data generation pipeline and fine-tuning a video generator.

Contribution

We introduce TrajectoryAtlas, a synthetic paired video data pipeline, and fine-tune a video generator to enable trajectory movement in videos, addressing data pairing challenges.

Findings

Successfully enables generative movement of object trajectories.

Uses TrajectoryAtlas for large-scale synthetic paired data.

Demonstrates effective trajectory editing in videos.

Abstract

Generative video editing has enabled several intuitive editing operations for short video clips that would previously have been difficult to achieve, especially for non-expert editors. Existing methods focus on prescribing an object's 3D or 2D motion trajectory in a video, or on altering the appearance of an object or a scene, while preserving both the video's plausibility and identity. Yet a method to move an object's 3D motion trajectory in a video, i.e., moving an object while preserving its relative 3D motion, is currently still missing. The main challenge lies in obtaining paired video data for this scenario. Previous methods typically rely on clever data generation approaches to construct plausible paired data from unpaired videos, but this approach fails if one of the videos in a pair can not easily be constructed from the other. Instead, we introduce TrajectoryAtlas, a new data generation pipeline for large-scale synthetic paired video data and a video generator TrajectoryMover fine-tuned with this data. We show that this successfully enables generative movement of object trajectories. Project page: https://chhatrekiran.github.io/trajectorymover