Classical Feature Embeddings Help in BERT-Based Human Mobility Prediction

TL;DR

This paper introduces STaBERT, a novel BERT-based model that incorporates POI and temporal data to enhance human mobility prediction accuracy across cities.

Contribution

The paper presents STaBERT, a model that effectively integrates semantic POI and temporal information into BERT for improved mobility forecasting.

Findings

Significant increase in GEO-BLEU score for single-city prediction (0.34 to 0.75)

Enhanced multi-city prediction accuracy (0.34 to 0.56)

Demonstrates the effectiveness of semantic enrichment in mobility models

Abstract

Human mobility forecasting is crucial for disaster relief, city planning, and public health. However, existing models either only model location sequences or include time information merely as auxiliary input, thereby failing to leverage the rich semantic context provided by points of interest (POIs). To address this, we enrich a BERT-based mobility model with derived temporal descriptors and POI embeddings to better capture the semantics underlying human movement. We propose STaBERT (Semantic-Temporal aware BERT), which integrates both POI and temporal information at each location to construct a unified, semantically enriched representation of mobility. Experimental results show that STaBERT significantly improves prediction accuracy: for single-city prediction, the GEO-BLEU score improved from 0.34 to 0.75; for multi-city prediction, from 0.34 to 0.56.