ATI: Any Trajectory Instruction for Controllable Video Generation
Angtian Wang, Haibin Huang, Jacob Zhiyuan Fang, Yiding Yang, Chongyang Ma

TL;DR
This paper introduces a unified framework for controllable video generation that integrates multiple motion types through trajectory-based inputs, enabling precise and semantically aligned motion control in generated videos.
Contribution
It presents a novel motion control method that projects user-defined trajectories into the latent space of pre-trained models, unifying camera, object, and local motions in a single framework.
Findings
Outperforms prior methods in controllability and visual quality
Supports diverse motion control tasks including stylized effects and viewpoint changes
Compatible with various state-of-the-art video generation models
Abstract
We propose a unified framework for motion control in video generation that seamlessly integrates camera movement, object-level translation, and fine-grained local motion using trajectory-based inputs. In contrast to prior methods that address these motion types through separate modules or task-specific designs, our approach offers a cohesive solution by projecting user-defined trajectories into the latent space of pre-trained image-to-video generation models via a lightweight motion injector. Users can specify keypoints and their motion paths to control localized deformations, entire object motion, virtual camera dynamics, or combinations of these. The injected trajectory signals guide the generative process to produce temporally consistent and semantically aligned motion sequences. Our framework demonstrates superior performance across multiple video motion control tasks, including…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Human Motion and Animation · 3D Shape Modeling and Analysis
