Multi-Agent AI Framework for Road Situation Detection and C-ITS Message Generation

TL;DR

This paper presents a multi-agent AI framework integrating multimodal large language models and vision-based perception for road-situation detection and C-ITS message generation, aiming to improve semantic understanding and reliability in traffic scenarios.

Contribution

It introduces a novel multi-agent AI system combining MLLMs with vision perception for enhanced road-situation monitoring and message generation in intelligent transportation systems.

Findings

Achieved 100% recall in situation detection

Generated correct C-ITS message schemas

Gemini-2.5-Flash underperforms Gemini-2.0-Flash in accuracy

Abstract

Conventional road-situation detection methods achieve strong performance in predefined scenarios but fail in unseen cases and lack semantic interpretation, which is crucial for reliable traffic recommendations. This work introduces a multi-agent AI framework that combines multimodal large language models (MLLMs) with vision-based perception for road-situation monitoring. The framework processes camera feeds and coordinates dedicated agents for situation detection, distance estimation, decision-making, and Cooperative Intelligent Transport System (C-ITS) message generation. Evaluation is conducted on a custom dataset of 103 images extracted from 20 videos of the TAD dataset. Both Gemini-2.0-Flash and Gemini-2.5-Flash were evaluated. The results show 100\% recall in situation detection and perfect message schema correctness; however, both models suffer from false-positive detections and have reduced performance in terms of number of lanes, driving lane status and cause code. Surprisingly, Gemini-2.5-Flash, though more capable in general tasks, underperforms Gemini-2.0-Flash in detection accuracy and semantic understanding and incurs higher latency (Table II). These findings motivate further work on fine-tuning specialized LLMs or MLLMs tailored for intelligent transportation applications.