Pretrained Mobility Transformer: A Foundation Model for Human Mobility

TL;DR

This paper introduces the Pretrained Mobility Transformer (PMT), a foundation model that leverages transformer architecture to analyze large-scale human mobility data, enabling improved predictions and understanding of urban spatial dynamics.

Contribution

The study presents the first application of transformer models to human mobility data, creating a foundation model that captures complex spatial-temporal patterns from unlabeled trajectory sequences.

Findings

PMT outperforms existing models in next-location prediction.

PMT effectively imputes missing trajectory data.

PMT generates realistic human mobility trajectories.

Abstract

Ubiquitous mobile devices are generating vast amounts of location-based service data that reveal how individuals navigate and utilize urban spaces in detail. In this study, we utilize these extensive, unlabeled sequences of user trajectories to develop a foundation model for understanding urban space and human mobility. We introduce the \textbf{P}retrained \textbf{M}obility \textbf{T}ransformer (PMT), which leverages the transformer architecture to process user trajectories in an autoregressive manner, converting geographical areas into tokens and embedding spatial and temporal information within these representations. Experiments conducted in three U.S. metropolitan areas over a two-month period demonstrate PMT's ability to capture underlying geographic and socio-demographic characteristics of regions. The proposed PMT excels across various downstream tasks, including next-location prediction, trajectory imputation, and trajectory generation. These results support PMT's capability and effectiveness in decoding complex patterns of human mobility, offering new insights into urban spatial functionality and individual mobility preferences.