StarNet: Joint Action-Space Prediction with Star Graphs and Implicit Global Frame Self-Attention

TL;DR

StarNet introduces a novel graph-based architecture for multi-agent trajectory prediction, effectively modeling map topology and agent interactions, and extends to joint scene prediction with improved accuracy on real-world datasets.

Contribution

It presents a new star graph-based map modeling approach and extends to joint multi-agent prediction using masked self-attention, enhancing scene understanding.

Findings

Achieves state-of-the-art results on inD and INTERACTION datasets.

Effectively models map topology with star graphs for better prediction.

Joint-StarNet improves multi-agent trajectory forecasts.

Abstract

In this work, we present a novel multi-modal multi-agent trajectory prediction architecture, focusing on map and interaction modeling using graph representation. For the purposes of map modeling, we capture rich topological structure into vector-based star graphs, which enable an agent to directly attend to relevant regions along polylines that are used to represent the map. We denote this architecture StarNet, and integrate it in a single-agent prediction setting. As the main result, we extend this architecture to joint scene-level prediction, which produces multiple agents' predictions simultaneously. The key idea in joint-StarNet is integrating the awareness of one agent in its own reference frame with how it is perceived from the points of view of other agents. We achieve this via masked self-attention. Both proposed architectures are built on top of the action-space prediction framework introduced in our previous work, which ensures kinematically feasible trajectory predictions. We evaluate the methods on the interaction-rich inD and INTERACTION datasets, with both StarNet and joint-StarNet achieving improvements over state of the art.