Trajeglish: Traffic Modeling as Next-Token Prediction

TL;DR

This paper introduces Trajeglish, a traffic scenario modeling approach using token prediction with a GPT-like model, achieving state-of-the-art realism and interaction metrics in traffic simulation.

Contribution

It presents a novel discrete tokenization scheme and autoregressive modeling framework for multi-agent traffic scenarios, surpassing prior benchmarks in realism and interaction.

Findings

Outperforms previous models on Waymo Sim Agents Benchmark

Shows adaptability of learned representations to nuScenes data

Analyzes the impact of context length and intra-timestep interactions

Abstract

A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. In pursuit of this functionality, we apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Using a simple data-driven tokenization scheme, we discretize trajectories to centimeter-level resolution using a small vocabulary. We then model the multi-agent sequence of discrete motion tokens with a GPT-like encoder-decoder that is autoregressive in time and takes into account intra-timestep interaction between agents. Scenarios sampled from our model exhibit state-of-the-art realism; our model tops the Waymo Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%. We ablate our modeling choices in full autonomy and partial autonomy settings, and show that the representations learned by our model can quickly be adapted to improve performance on nuScenes. We additionally evaluate the scalability of our model with respect to parameter count and dataset size, and use density estimates from our model to quantify the saliency of context length and intra-timestep interaction for the traffic modeling task.