Learning Structured Representations of Spatial and Interactive Dynamics for Trajectory Prediction in Crowded Scenes

TL;DR

This paper introduces a modular, learned environment model for trajectory prediction that adapts unsupervised to new scenes and tasks, improving robustness and reducing the need for re-training.

Contribution

It presents a novel modular approach that models spatial and dynamic environment aspects for unsupervised transfer across tasks and environments.

Findings

Achieves performance comparable to state-of-the-art in pedestrian prediction

Successfully transfers predictor across pedestrian and robot tasks without labels

Enables robust, label-efficient forward modeling in new environments

Abstract

Context plays a significant role in the generation of motion for dynamic agents in interactive environments. This work proposes a modular method that utilises a learned model of the environment for motion prediction. This modularity explicitly allows for unsupervised adaptation of trajectory prediction models to unseen environments and new tasks by relying on unlabelled image data only. We model both the spatial and dynamic aspects of a given environment alongside the per agent motions. This results in more informed motion prediction and allows for performance comparable to the state-of-the-art. We highlight the model's prediction capability using a benchmark pedestrian prediction problem and a robot manipulation task and show that we can transfer the predictor across these tasks in a completely unsupervised way. The proposed approach allows for robust and label efficient forward modelling, and relaxes the need for full model re-training in new environments.