Natural Adversaries: Fuzzing Autonomous Vehicles with Realistic Roadside Object Placements

TL;DR

This paper introduces TrashFuzz, a black-box fuzzing method that creates realistic roadside object placements to test autonomous vehicle perception systems, revealing vulnerabilities that cause traffic law violations.

Contribution

The work presents a novel approach to generate realistic adversarial scenarios for AVs by manipulating roadside object positions according to regulatory guidelines, without using unnatural patches.

Findings

Induced 15 out of 24 traffic law violations in tests

Created realistic adversarial scenarios respecting road design rules

Demonstrated effectiveness on Apollo autonomous driving system

Abstract

The emergence of Autonomous Vehicles (AVs) has spurred research into testing the resilience of their perception systems, i.e., ensuring that they are not susceptible to critical misjudgements. It is important that these systems are tested not only with respect to other vehicles on the road, but also with respect to objects placed on the roadside. Trash bins, billboards, and greenery are examples of such objects, typically positioned according to guidelines developed for the human visual system, which may not align perfectly with the needs of AVs. Existing tests, however, usually focus on adversarial objects with conspicuous shapes or patches, which are ultimately unrealistic due to their unnatural appearance and reliance on white-box knowledge. In this work, we introduce a black-box attack on AV perception systems that creates realistic adversarial scenarios (i.e., satisfying road design guidelines) by manipulating the positions of common roadside objects and without resorting to "unnatural" adversarial patches. In particular, we propose TrashFuzz, a fuzzing algorithm that finds scenarios in which the placement of these objects leads to substantial AV misperceptions -- such as mistaking a traffic light's colour -- with the overall goal of causing traffic-law violations. To ensure realism, these scenarios must satisfy several rules encoding regulatory guidelines governing the placement of objects on public streets. We implemented and evaluated these attacks on the Apollo autonomous driving system, finding that TrashFuzz induced violations of 15 out of 24 traffic laws.