AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points

TL;DR

AutoTrajectory introduces an unsupervised, label-free method for extracting and predicting trajectories directly from raw videos using dynamic points, eliminating the need for ground truth annotations.

Contribution

It is the first to achieve unsupervised learning of trajectory extraction and prediction from raw videos, leveraging dynamic points for motion modeling.

Findings

Effective on well-known trajectory datasets

Improves existing models by using raw videos

Demonstrates robustness in real-world scenarios

Abstract

Current methods for trajectory prediction operate in supervised manners, and therefore require vast quantities of corresponding ground truth data for training. In this paper, we present a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction to use raw videos directly. To better capture the moving objects in videos, we introduce dynamic points. We use them to model dynamic motions by using a forward-backward extractor to keep temporal consistency and using image reconstruction to keep spatial consistency in an unsupervised manner. Then we aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos. Finally, we extract trajectories by matching instance points for prediction training. To the best of our knowledge, our method is the first to achieve unsupervised learning of trajectory extraction and prediction. We evaluate the performance on well-known trajectory datasets and show that our method is effective for real-world videos and can use raw videos to further improve the performance of existing models.