LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehicles

TL;DR

LawBreaker is an automated framework that tests autonomous driving systems against traffic laws using a rich specification language and fuzzing, successfully identifying law violations and unsafe scenarios in simulations.

Contribution

It introduces a novel driver-oriented traffic law specification language and a fuzzing approach to systematically test ADS compliance with real-world laws.

Findings

Found 14 traffic law violations in simulations

Generated 173 test cases leading to accidents

Demonstrated effectiveness on Apollo+LGSVL with Chinese laws

Abstract

Autonomous driving systems (ADSs) must be tested thoroughly before they can be deployed in autonomous vehicles. High-fidelity simulators allow them to be tested against diverse scenarios, including those that are difficult to recreate in real-world testing grounds. While previous approaches have shown that test cases can be generated automatically, they tend to focus on weak oracles (e.g. reaching the destination without collisions) without assessing whether the journey itself was undertaken safely and satisfied the law. In this work, we propose LawBreaker, an automated framework for testing ADSs against real-world traffic laws, which is designed to be compatible with different scenario description languages. LawBreaker provides a rich driver-oriented specification language for describing traffic laws, and a fuzzing engine that searches for different ways of violating them by maximising specification coverage. To evaluate our approach, we implemented it for Apollo+LGSVL and specified the traffic laws of China. LawBreaker was able to find 14 violations of these laws, including 173 test cases that caused accidents.