ROMAN: Reward-Orchestrated Multi-Head Attention Network for Autonomous Driving System Testing

TL;DR

ROMAN is a novel scenario generation method for autonomous driving testing that uses multi-head attention and traffic law weighting to create high-risk, diverse violation scenarios, improving evaluation coverage.

Contribution

ROMAN introduces a multi-head attention network combined with a traffic law weighting mechanism for generating targeted, high-risk violation scenarios in ADS testing, addressing limitations of existing methods.

Findings

ROMAN achieved 7.91% higher violation count than ABLE.

ROMAN generated violation scenarios for all traffic law clauses.

ROMAN demonstrated greater scenario diversity and high-risk violation detection.

Abstract

Automated Driving System (ADS) acts as the brain of autonomous vehicles, responsible for their safety and efficiency. Safe deployment requires thorough testing in diverse real-world scenarios and compliance with traffic laws like speed limits, signal obedience, and right-of-way rules. Violations like running red lights or speeding pose severe safety risks. However, current testing approaches face significant challenges: limited ability to generate complex and high-risk law-breaking scenarios, and failing to account for complex interactions involving multiple vehicles and critical situations. To address these challenges, we propose ROMAN, a novel scenario generation approach for ADS testing that combines a multi-head attention network with a traffic law weighting mechanism. ROMAN is designed to generate high-risk violation scenarios to enable more thorough and targeted ADS evaluation. The multi-head attention mechanism models interactions among vehicles, traffic signals, and other factors. The traffic law weighting mechanism implements a workflow that leverages an LLM-based risk weighting module to evaluate violations based on the two dimensions of severity and occurrence. We have evaluated ROMAN by testing the Baidu Apollo ADS within the CARLA simulation platform and conducting extensive experiments to measure its performance. Experimental results demonstrate that ROMAN surpassed state-of-the-art tools ABLE and LawBreaker by achieving 7.91% higher average violation count than ABLE and 55.96% higher than LawBreaker, while also maintaining greater scenario diversity. In addition, only ROMAN successfully generated violation scenarios for every clause of the input traffic laws, enabling it to identify more high-risk violations than existing approaches.