TrajFM: A Vehicle Trajectory Foundation Model for Region and Task Transferability

TL;DR

TrajFM is a versatile vehicle trajectory foundation model designed for effective transfer across different regions and tasks, utilizing a novel architecture and training scheme to improve generalization and reduce retraining needs.

Contribution

The paper introduces TrajFM, a novel trajectory foundation model with STRFormer for region transferability and a masking scheme for task transferability, enabling one-time pre-training for multiple tasks.

Findings

TrajFM outperforms existing models in region transfer tasks.

The masking scheme enables effective multi-task transfer without retraining.

Experiments demonstrate TrajFM's superior generalization on real-world datasets.

Abstract

Vehicle trajectories provide valuable movement information that supports various downstream tasks and powers real-world applications. A desirable trajectory learning model should transfer between different regions and tasks without retraining, thus improving computational efficiency and effectiveness with limited training data. However, a model's ability to transfer across regions is limited by the unique spatial features and POI arrangements of each region, which are closely linked to vehicle movement patterns and difficult to generalize. Additionally, achieving task transferability is challenging due to the differing generation schemes required for various tasks. Existing efforts towards transferability primarily involve learning embedding vectors for trajectories, which perform poorly in region transfer and still require retraining of prediction modules for task transfer. To address these challenges, we propose TrajFM, a vehicle trajectory foundation model that excels in both region and task transferability. For region transferability, we introduce STRFormer as the main learnable model within TrajFM. It integrates spatial, temporal, and POI modalities of trajectories to effectively manage variations in POI arrangements across regions and includes a learnable spatio-temporal Rotary position embedding module for handling spatial features. For task transferability, we propose a trajectory masking and recovery scheme. This scheme unifies the generation processes of various tasks into the masking and recovery of modalities and sub-trajectories, allowing TrajFM to be pre-trained once and transferred to different tasks without retraining. Experiments on two real-world vehicle trajectory datasets under various settings demonstrate the effectiveness of TrajFM. Code is available at https://anonymous.4open.science/r/TrajFM-30E4.