Applying Large Language Models to Travel Satisfaction Analysis

TL;DR

This paper explores the use of Large Language Models (LLMs) for modeling travel satisfaction, demonstrating that few-shot learning can improve prediction accuracy with small datasets, addressing limitations of traditional statistical and machine learning methods.

Contribution

It introduces an LLM-based approach for travel satisfaction analysis that effectively handles small sample sizes through few-shot learning, highlighting its potential for behavioral modeling.

Findings

Zero-shot LLM shows behavioral misalignment and low accuracy.

Few-shot learning aligns LLMs and outperforms baseline models.

Discrepancies in variable importance reveal dataset-specific gaps.

Abstract

As a specific domain of subjective well-being, travel satisfaction has recently attracted much research attention. Previous studies primarily relied on statistical models and, more recently, machine learning models to explore its determinants. Both approaches,however, depend on sufficiently large sample sizes and appropriate statistical assumptions. The emergence of Large Language Models (LLMs) offers a new modeling approach that can address these limitations. Pre-trained on extensive datasets, LLMs have strongcapabilities in contextual understanding and generalization, significantly reducing their dependence on task-specific data and stringent statistical assumptions. The main challenge in applying LLMs lies in the behavioral misalignment between LLMs and humans. Using household survey data collected in Shanghai, this study identifies the existence and source of misalignment, and applies a few-shot learning method to address the misalignment issue. We find that the zero-shot LLM exhibits behavioral misalignment, leading to low prediction accuracy. With just a few samples, few-shot learning can align LLMs and enable them to outperform baseline models. Discrepancies in variable importance among machine learning model, zero-shot LLM, and few-shot LLM reveal that the misalignment arises from the gap between the general knowledge embedded in pre-trained LLMs and the specific, unique characteristics of the dataset. On these bases, we propose an LLM-based modeling approach that can be applied to model travel behavior with small sample sizes. This study highlights the potential of LLMs for modeling not only travel satisfaction but also broader aspects of travel behavior.