Point-to-Point: Sparse Motion Guidance for Controllable Video Editing

TL;DR

Point-to-Point introduces anchor tokens, a novel motion representation leveraging diffusion models, enabling controllable, semantically aligned video editing with improved motion fidelity across diverse scenarios.

Contribution

The paper proposes anchor tokens, a new motion representation that captures essential video dynamics and enhances controllability in video editing tasks.

Findings

Achieves superior edit and motion fidelity compared to existing methods.

Enables flexible relocation of motion patterns to new subjects.

Demonstrates generalization across diverse video scenarios.

Abstract

Accurately preserving motion while editing a subject remains a core challenge in video editing tasks. Existing methods often face a trade-off between edit and motion fidelity, as they rely on motion representations that are either overfitted to the layout or only implicitly defined. To overcome this limitation, we revisit point-based motion representation. However, identifying meaningful points remains challenging without human input, especially across diverse video scenarios. To address this, we propose a novel motion representation, anchor tokens, that capture the most essential motion patterns by leveraging the rich prior of a video diffusion model. Anchor tokens encode video dynamics compactly through a small number of informative point trajectories and can be flexibly relocated to align with new subjects. This allows our method, Point-to-Point, to generalize across diverse scenarios. Extensive experiments demonstrate that anchor tokens lead to more controllable and semantically aligned video edits, achieving superior performance in terms of edit and motion fidelity.