Algorithmic Scenario Generation as Quality Diversity Optimization

TL;DR

This paper reviews a framework for generating diverse, realistic test scenarios for complex robots and autonomous agents to identify failures before deployment, enhancing safety and reliability.

Contribution

It introduces a general framework for scenario generation using quality diversity optimization, integrating components to discover challenging scenarios revealing system failures.

Findings

Framework effectively generates diverse test scenarios

Reveals previously unknown system failures

Enhances robot safety and reliability

Abstract

The increasing complexity of robots and autonomous agents that interact with people highlights the critical need for approaches that systematically test them before deployment. This review paper presents a general framework for solving this problem, describes the insights that we have gained from working on each component of the framework, and shows how integrating these components leads to the discovery of a diverse range of realistic and challenging scenarios that reveal previously unknown failures in deployed robotic systems interacting with people.