Driving Style Recognition Like an Expert Using Semantic Privileged Information from Large Language Models

TL;DR

This paper introduces a novel driving style recognition framework that leverages semantic privileged information from large language models to improve interpretability and accuracy, aligning algorithmic outputs with human expert judgments.

Contribution

The authors propose DriBehavGPT and an SVM+ based approach to incorporate semantic descriptions from LLMs as privileged information during training, enhancing recognition performance.

Findings

SPI improves F1-score by up to 7.9% in driving scenarios.

The framework achieves better alignment with human reasoning.

Inference remains efficient by using only sensor data.

Abstract

Existing driving style recognition systems largely depend on low-level sensor-derived features for training, neglecting the rich semantic reasoning capability inherent to human experts. This discrepancy results in a fundamental misalignment between algorithmic classifications and expert judgments. To bridge this gap, we propose a novel framework that integrates Semantic Privileged Information (SPI) derived from large language models (LLMs) to align recognition outcomes with human-interpretable reasoning. First, we introduce DriBehavGPT, an interactive LLM-based module that generates natural-language descriptions of driving behaviors. These descriptions are then encoded into machine learning-compatible representations via text embedding and dimensionality reduction. Finally, we incorporate them as privileged information into Support Vector Machine Plus (SVM+) for training, enabling the model to approximate human-like interpretation patterns. Experiments across diverse real-world driving scenarios demonstrate that our SPI-enhanced framework outperforms conventional methods, achieving F1-score improvements of 7.6% (car-following) and 7.9% (lane-changing). Importantly, SPI is exclusively used during training, while inference relies solely on sensor data, ensuring computational efficiency without sacrificing performance. These results highlight the pivotal role of semantic behavioral representations in improving recognition accuracy while advancing interpretable, human-centric driving systems.