LIFT: Interpretable truck driving risk prediction with literature-informed fine-tuned LLMs

TL;DR

This paper introduces LIFT, an interpretable truck driving risk prediction framework using literature-informed fine-tuned LLMs that outperform benchmarks and provide explainable insights into risk factors.

Contribution

The study presents a novel framework combining literature processing and fine-tuned LLMs for interpretable risk prediction in truck driving, with improved accuracy and explainability.

Findings

LIFT outperforms benchmark models by 26.7% in recall and 10.1% in F1-score.

The literature knowledge base aligns variable importance with benchmark models.

LIFT identifies risky scenarios verified by statistical tests.

Abstract

This study proposes an interpretable prediction framework with literature-informed fine-tuned (LIFT) LLMs for truck driving risk prediction. The framework integrates an LLM-driven Inference Core that predicts and explains truck driving risk, a Literature Processing Pipeline that filters and summarizes domain-specific literature into a literature knowledge base, and a Result Evaluator that evaluates the prediction performance as well as the interpretability of the LIFT LLM. After fine-tuning on a real-world truck driving risk dataset, the LIFT LLM achieved accurate risk prediction, outperforming benchmark models by 26.7% in recall and 10.1% in F1-score. Furthermore, guided by the literature knowledge base automatically constructed from 299 domain papers, the LIFT LLM produced variable importance ranking consistent with that derived from the benchmark model, while demonstrating robustness in interpretation results to various data sampling conditions. The LIFT LLM also identified potential risky scenarios by detecting key combination of variables in truck driving risk, which were verified by PERMANOVA tests. Finally, we demonstrated the contribution of the literature knowledge base and the fine-tuning process in the interpretability of the LIFT LLM, and discussed the potential of the LIFT LLM in data-driven knowledge discovery.