Assessing behaviour coverage in a multi-agent system simulation for autonomous vehicle testing

TL;DR

This paper presents a systematic approach to measure and improve behaviour coverage in multi-agent system simulations for autonomous vehicle testing, aiming to enhance safety and reliability in diverse scenarios.

Contribution

It introduces a behaviour coverage assessment framework and proposes a Model Predictive Control pedestrian agent to generate more realistic and interesting test scenarios.

Findings

Behaviour coverage metrics identify key areas for simulation improvement

Coverage-based testing enhances autonomous vehicle validation

MPC pedestrian agent promotes realistic and diverse behaviours

Abstract

As autonomous vehicle technology advances, ensuring the safety and reliability of these systems becomes paramount. Consequently, comprehensive testing methodologies are essential to evaluate the performance of autonomous vehicles in diverse and complex real-world scenarios. This study focuses on the behaviour coverage analysis of a multi-agent system simulation designed for autonomous vehicle testing, and provides a systematic approach to measure and assess behaviour coverage within the simulation environment. By defining a set of driving scenarios, and agent interactions, we evaluate the extent to which the simulation encompasses a broad range of behaviours relevant to autonomous driving. Our findings highlight the importance of behaviour coverage in validating the effectiveness and robustness of autonomous vehicle systems. Through the analysis of behaviour coverage metrics and coverage-based testing, we identify key areas for improvement and optimization in the simulation framework. Thus, a Model Predictive Control (MPC) pedestrian agent is proposed, where its objective function is formulated to encourage \textit{interesting} tests while promoting a more realistic behaviour than other previously studied pedestrian agents. This research contributes to advancing the field of autonomous vehicle testing by providing insights into the comprehensive evaluation of system behaviour in simulated environments. The results offer valuable implications for enhancing the safety, reliability, and performance of autonomous vehicles through rigorous testing methodologies.