TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding

TL;DR

TrajPrism is a comprehensive benchmark that evaluates the alignment between urban trajectories and natural language descriptions across multiple tasks, promoting integrated research in language-grounded mobility understanding.

Contribution

It introduces a novel multi-task benchmark with a large, diverse dataset and baseline models for trajectory generation, retrieval, and captioning, unifying geometry and language modalities.

Findings

Geometry-only baselines perform poorly on language-grounded tasks.

The benchmark contains 300K trajectories with 2.1M task instances across three cities.

Baseline models show significant room for improvement in language-trajectory alignment.

Abstract

Urban mobility is naturally expressed both as trajectories in space and as natural-language descriptions of travel intent, constraints, and preferences. However, prior work rarely evaluates these two modalities together on the same real-world trajectories: trajectory modeling often stays geometry-centric, while language-centric mobility benchmarks frequently target route planning and tool use rather than fine-grained, verifiable alignment between text and the underlying route. We introduce TrajPrism, a multi-task benchmark for language-trajectory alignment that unifies (i) instruction-conditioned trajectory generation, (ii) language-driven semantic trajectory retrieval, and (iii) trajectory captioning, together with an evaluation protocol that measures trajectory fidelity, retrieval quality, and language groundedness. We construct TrajPrism by pairing real urban trajectories with judge-filtered language annotations generated under a four-dimensional travel-intent taxonomy. The benchmark contains 300K selected trajectories across Porto, San Francisco, and Beijing, yielding 2.1M task instances from three instruction variants, three retrieval queries, and one caption per trajectory. We further develop proof-of-concept models for each task: TrajAnchor for instruction-conditioned trajectory generation, TrajFuse for semantic trajectory retrieval, and TrajRap for trajectory captioning. These models instantiate the proposed tasks and show that geometry-only trajectory baselines leave a large gap on our protocol, especially where language is part of the input-output interface. We release TrajPrism with code and a reproducible annotation pipeline that is designed to be portable across cities, given compatible trajectory inputs and map resources.