Reinforcement Learning for Testing Interdependent Requirements in Autonomous Vehicles: An Empirical Study

TL;DR

This study compares single-objective and multi-objective reinforcement learning methods for generating test scenarios in autonomous vehicle testing, highlighting their differences in violation detection and scenario diversity.

Contribution

It provides an empirical comparison of SORL and MORL in testing interdependent requirements of autonomous vehicles, emphasizing the importance of considering requirement dependencies.

Findings

MORL generates more requirement-violation scenarios.

SORL produces higher-severity violations.

MORL covers a broader range of scenarios.

Abstract

Autonomous vehicles (AVs) make driving decisions without humans, making dependability assurance critical. Scenario-based testing is widely used to evaluate AVs under diverse conditions, with reinforcement learning (RL) generating test scenarios that identify violations of functional and safety requirements. Many requirements are interdependent and involve trade-offs, making it unclear whether single-objective RL (SORL), which combines objectives into a single reward, can reliably reveal violations or whether multi-objective RL (MORL), which explicitly considers multiple objectives, is necessary. We present an empirical evaluation comparing SORL and MORL for generating critical scenarios that simultaneously test interdependent requirements using an end-to-end AV controller and high-fidelity simulator. Results suggest that MORL and SORL differ mainly in how violations occur, while showing comparable effectiveness in many cases. MORL tends to generate more requirement-violation scenarios, whereas SORL produces higher-severity violations. Their relative performance also depends on specific objective combinations and, to a lesser extent, road conditions. Regarding diversity, MORL consistently covers a broader range of scenarios. Thus, MORL is preferable when scenario diversity and coverage are prioritized, whereas SORL may better expose severe violations. Our empirical evaluation addresses a gap by systematically comparing SORL and MORL, highlighting the importance of requirement dependencies in RL-based AV testing.