BERT4Traj: Transformer Based Trajectory Reconstruction for Sparse Mobility Data

TL;DR

BERT4Traj is a transformer-based model designed to reconstruct complete human mobility trajectories from sparse data, significantly improving over traditional methods by leveraging spatial, temporal, and contextual features.

Contribution

We introduce BERT4Traj, a novel transformer model that effectively reconstructs mobility trajectories from sparse data using masked modeling and contextual embeddings.

Findings

BERT4Traj outperforms Markov Chains, KNN, RNNs, and LSTMs in trajectory reconstruction.

The model effectively captures detailed and continuous mobility patterns.

Results are validated on real-world CDR and GPS datasets from Kampala, Uganda.

Abstract

Understanding human mobility is essential for applications in public health, transportation, and urban planning. However, mobility data often suffers from sparsity due to limitations in data collection methods, such as infrequent GPS sampling or call detail record (CDR) data that only capture locations during communication events. To address this challenge, we propose BERT4Traj, a transformer based model that reconstructs complete mobility trajectories by predicting hidden visits in sparse movement sequences. Inspired by BERT's masked language modeling objective and self_attention mechanisms, BERT4Traj leverages spatial embeddings, temporal embeddings, and contextual background features such as demographics and anchor points. We evaluate BERT4Traj on real world CDR and GPS datasets collected in Kampala, Uganda, demonstrating that our approach significantly outperforms traditional models such as Markov Chains, KNN, RNNs, and LSTMs. Our results show that BERT4Traj effectively reconstructs detailed and continuous mobility trajectories, enhancing insights into human movement patterns.