Encoding Agent Trajectories as Representations with Sequence Transformers

TL;DR

This paper introduces STARE, a Transformer-based model that encodes high-dimensional spatiotemporal trajectories into meaningful representations, enabling improved analysis and understanding of agent movements and relationships in space and time.

Contribution

The paper presents a novel Transformer architecture for encoding spatiotemporal trajectories as sequences, applying both supervised and self-supervised learning to capture structure and relationships.

Findings

Effective in learning meaningful trajectory representations

Improves downstream tasks like classification and similarity detection

Captures relationships between agents and locations

Abstract

Spatiotemporal data faces many analogous challenges to natural language text including the ordering of locations (words) in a sequence, long range dependencies between locations, and locations having multiple meanings. In this work, we propose a novel model for representing high dimensional spatiotemporal trajectories as sequences of discrete locations and encoding them with a Transformer-based neural network architecture. Similar to language models, our Sequence Transformer for Agent Representation Encodings (STARE) model can learn representations and structure in trajectory data through both supervisory tasks (e.g., classification), and self-supervisory tasks (e.g., masked modelling). We present experimental results on various synthetic and real trajectory datasets and show that our proposed model can learn meaningful encodings that are useful for many downstream tasks including discriminating between labels and indicating similarity between locations. Using these encodings, we also learn relationships between agents and locations present in spatiotemporal data.