Dual-Stage LLM Framework for Scenario-Centric Semantic Interpretation in Driving Assistance

TL;DR

This paper introduces a scenario-centric framework for auditing LLM-based risk reasoning in urban driving, highlighting semantic ambiguities and model divergences in safety-critical assessments.

Contribution

It proposes a structured, reproducible method for evaluating LLMs in driving scenarios, emphasizing the importance of ambiguity management and scenario-based analysis.

Findings

Models show systematic divergence in risk severity and escalation.

Semantic indeterminacy often causes variability in interpretations.

Explicit ambiguity handling is crucial for safety-aligned driver assistance.

Abstract

Advanced Driver Assistance Systems (ADAS) increasingly rely on learning-based perception, yet safety-relevant failures often arise without component malfunction, driven instead by partial observability and semantic ambiguity in how risk is interpreted and communicated. This paper presents a scenario-centric framework for reproducible auditing of LLM-based risk reasoning in urban driving contexts. Deterministic, temporally bounded scenario windows are constructed from multimodal driving data and evaluated under fixed prompt constraints and a closed numeric risk schema, ensuring structured and comparable outputs across models. Experiments on a curated near-people scenario set compare two text-only models and one multimodal model under identical inputs and prompts. Results reveal systematic inter-model divergence in severity assignment, high-risk escalation, evidence use, and causal attribution. Disagreement extends to the interpretation of vulnerable road user presence, indicating that variability often reflects intrinsic semantic indeterminacy rather than isolated model failure. These findings highlight the importance of scenario-centric auditing and explicit ambiguity management when integrating LLM-based reasoning into safety-aligned driver assistance systems.