Towards Locally Deployable Fine-Tuned Causal Large Language Models for Mode Choice Behaviour

TL;DR

This paper introduces LiTransMC, a fine-tuned causal large language model for travel mode choice prediction, demonstrating high accuracy, interpretability, and local deployability, advancing transportation modeling and policy analysis.

Contribution

It presents the first fine-tuned causal LLM for mode choice, benchmarking multiple models, and combining predictive accuracy with interpretability for transportation research.

Findings

LiTransMC outperforms larger proprietary models and classical methods.

Achieves high predictive accuracy with F1 score of 0.6845.

Provides structured analysis of decision factors aligned with behavioral theory.

Abstract

This study investigates the adoption of open-access, locally deployable causal large language models (LLMs) for travel mode choice prediction and introduces LiTransMC, the first fine-tuned causal LLM developed for this task. We systematically benchmark eleven open-access LLMs (1-12B parameters) across three stated and revealed preference datasets, testing 396 configurations and generating over 79,000 mode choice decisions. Beyond predictive accuracy, we evaluate models generated reasoning using BERTopic for topic modelling and a novel Explanation Strength Index, providing the first structured analysis of how LLMs articulate decision factors in alignment with behavioural theory. LiTransMC, fine-tuned using parameter efficient and loss masking strategy, achieved a weighted F1 score of 0.6845 and a Jensen-Shannon Divergence of 0.000245, surpassing both untuned local models and larger proprietary systems, including GPT-4o with advanced persona inference and embedding-based loading, while also outperforming classical mode choice methods such as discrete choice models and machine learning classifiers for the same dataset. This dual improvement, i.e., high instant-level accuracy and near-perfect distributional calibration, demonstrates the feasibility of creating specialist, locally deployable LLMs that integrate prediction and interpretability. Through combining structured behavioural prediction with natural language reasoning, this work unlocks the potential for conversational, multi-task transport models capable of supporting agent-based simulations, policy testing, and behavioural insight generation. These findings establish a pathway for transforming general purpose LLMs into specialized and explainable tools for transportation research and policy formulation, while maintaining privacy, reducing cost, and broadening access through local deployment.