Tracking Everything Everywhere All at Once
Qianqian Wang, Yen-Yu Chang, Ruojin Cai, Zhengqi Li, Bharath, Hariharan, Aleksander Holynski, Noah Snavely

TL;DR
This paper introduces OmniMotion, a novel test-time optimization method that achieves globally consistent, dense, and long-range motion estimation for entire videos, overcoming limitations of prior methods in occlusion handling and temporal consistency.
Contribution
OmniMotion is a new comprehensive motion representation that enables accurate, full-length pixel-wise motion tracking across entire videos, ensuring global consistency and occlusion robustness.
Findings
Outperforms prior methods on TAP-Vid benchmark
Achieves accurate long-range motion estimation
Handles occlusions effectively
Abstract
We present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Tracking Everything Everywhere All at Once· youtube
Taxonomy
TopicsAdvanced Vision and Imaging · Advanced Image Processing Techniques · Robotics and Sensor-Based Localization
