# Situational perception in distracted driving: an agentic multi-modal LLM framework

## TL;DR

This paper introduces a new AI system that helps distracted drivers by providing real-time alerts based on camera and GPS data.

## Contribution

The novel contribution is an agentic multi-modal LLM framework for real-time, context-aware driver alerts during distraction.

## Key findings

- The system achieves 85.7% semantic correctness in driver alerts.
- Average response latency is 1.74 seconds, meeting real-time safety requirements.
- The framework outperforms conventional ML-based systems in synthesizing multi-modal data.

## Abstract

Distracted driving is a significant public safety concern, causing thousands of accidents annually. While most driver assistance systems emphasize distraction detection, they fail to deliver real-time environmental perception and context-aware interventions.

We propose a large language model (LLM)-driven intervention framework that assumes distraction is pre-detected and dynamically integrates camera and GPS inputs to generate verbal driver alerts. The framework employs an agentic design, where specialized tools handle object detection, speed limits, live traffic conditions, and weather data. Structured orchestration ensures information is fused efficiently, balancing accuracy with conciseness to avoid overwhelming the driver.

Evaluation of the system demonstrates high performance, with semantic intervention correctness of 85.7% and an average response latency of 1.74 s. Compared to conventional ML-based driver assistance approaches, our framework effectively synthesizes multi-modal environmental data and produces actionable alerts in real time.

These findings highlight the potential of LLM-driven, multi-modal reasoning for distracted driving intervention. Integrating specialized agents and structured orchestration improves situational awareness, maintains concise communication, and meets real-time safety requirements. This proof-of-concept establishes a pathway for deploying intelligent, AI-driven driver support systems in safety-critical applications.