TranSPORTmer: A Holistic Approach to Trajectory Understanding in Multi-Agent Sports

TL;DR

TranSPORTmer is a unified transformer-based framework that comprehensively addresses trajectory prediction, missing data imputation, agent state inference, and global state classification in multi-agent sports scenarios, outperforming specialized models.

Contribution

It introduces a holistic transformer model with Set Attention Blocks and a CLS agent for multiple trajectory understanding tasks in sports, unifying several tasks into one framework.

Findings

Outperforms state-of-the-art models in player forecasting and imputation.

Effectively captures social interactions and temporal dynamics.

Enhances trajectory modeling with a dedicated classification agent.

Abstract

Understanding trajectories in multi-agent scenarios requires addressing various tasks, including predicting future movements, imputing missing observations, inferring the status of unseen agents, and classifying different global states. Traditional data-driven approaches often handle these tasks separately with specialized models. We introduce TranSPORTmer, a unified transformer-based framework capable of addressing all these tasks, showcasing its application to the intricate dynamics of multi-agent sports scenarios like soccer and basketball. Using Set Attention Blocks, TranSPORTmer effectively captures temporal dynamics and social interactions in an equivariant manner. The model's tasks are guided by an input mask that conceals missing or yet-to-be-predicted observations. Additionally, we introduce a CLS extra agent to classify states along soccer trajectories, including passes, possessions, uncontrolled states, and out-of-play intervals, contributing to an enhancement in modeling trajectories. Evaluations on soccer and basketball datasets show that TranSPORTmer outperforms state-of-the-art task-specific models in player forecasting, player forecasting-imputation, ball inference, and ball imputation. https://youtu.be/8VtSRm8oGoE