Evaluating the Effectiveness of Large Language Models in Representing and Understanding Movement Trajectories

TL;DR

This paper evaluates how well large language models like GPT-J can represent and analyze movement trajectories, highlighting their strengths in understanding spatiotemporal dependencies and limitations in numeric and spatial retrieval tasks.

Contribution

It demonstrates the potential and challenges of using LLMs for trajectory data analysis and emphasizes the need for domain-specific enhancements.

Findings

LLMs preserve certain trajectory distance metrics with high correlation

LLMs can understand spatiotemporal dependencies and predict locations accurately

Challenges remain in numeric value restoration and spatial neighbor retrieval

Abstract

This research focuses on assessing the ability of AI foundation models in representing the trajectories of movements. We utilize one of the large language models (LLMs) (i.e., GPT-J) to encode the string format of trajectories and then evaluate the effectiveness of the LLM-based representation for trajectory data analysis. The experiments demonstrate that while the LLM-based embeddings can preserve certain trajectory distance metrics (i.e., the correlation coefficients exceed 0.74 between the Cosine distance derived from GPT-J embeddings and the Hausdorff and Dynamic Time Warping distances on raw trajectories), challenges remain in restoring numeric values and retrieving spatial neighbors in movement trajectory analytics. In addition, the LLMs can understand the spatiotemporal dependency contained in trajectories and have good accuracy in location prediction tasks. This research highlights the need for improvement in terms of capturing the nuances and complexities of the underlying geospatial data and integrating domain knowledge to support various GeoAI applications using LLMs.