Learning Multi-Modal Mobility Dynamics for Generalized Next Location Recommendation

TL;DR

This paper introduces M3ob, a multi-modal mobility model that leverages spatial-temporal knowledge graphs and large language models to improve next location predictions and generalize across normal and abnormal scenarios.

Contribution

The paper proposes a novel multi-modal approach using a unified spatial-temporal graph and LLM-enhanced knowledge to better capture mobility dynamics for location recommendation.

Findings

Achieves consistent improvements across six datasets.

Demonstrates strong generalization in abnormal scenarios.

Effectively fuses multi-modal spatial-temporal information.

Abstract

The precise prediction of human mobility has produced significant socioeconomic impacts, such as location recommendations and evacuation suggestions. However, existing methods suffer from limited generalization capability: unimodal approaches are constrained by data sparsity and inherent biases, while multi-modal methods struggle to effectively capture mobility dynamics caused by the semantic gap between static multi-modal representation and spatial-temporal dynamics. Therefore, we leverage multi-modal spatial-temporal knowledge to characterize mobility dynamics for the location recommendation task, dubbed as \textbf{M}ulti-\textbf{M}odal \textbf{Mob}ility (\textbf{M}$^3$\textbf{ob}). First, we construct a unified spatial-temporal relational graph (STRG) for multi-modal representation, by leveraging the functional semantics and spatial-temporal knowledge captured by the large language models (LLMs)-enhanced spatial-temporal knowledge graph (STKG). Second, we design a gating mechanism to fuse spatial-temporal graph representations of different modalities, and propose an STKG-guided cross-modal alignment to inject spatial-temporal dynamic knowledge into the static image modality. Extensive experiments on six public datasets show that our proposed method not only achieves consistent improvements in normal scenarios but also exhibits significant generalization ability in abnormal scenarios.