Dynamic Deception: When Pedestrians Team Up to Fool Autonomous Cars

TL;DR

This paper demonstrates that coordinated, dynamic adversarial attacks by multiple pedestrians can cause autonomous vehicles to fail at the system level, highlighting vulnerabilities in end-to-end safety that are not apparent at the model level.

Contribution

It introduces a novel system-level attack involving multiple pedestrians with adversarial patches that coordinate and move to fool autonomous driving systems.

Findings

Single-pedestrian attacks fail in all runs.

Two pedestrians with coordinated movement cause vehicle stops in up to 50% of runs.

Static collusion does not successfully attack the system.

Abstract

Many adversarial attacks on autonomous-driving perception models fail to cause system-level failures once deployed in a full driving stack. The main reason for such ineffectiveness is that once deployed in a system (e.g., within a simulator), attacks tend to be spatially or temporally short-lived, due to the vehicle's dynamics, hence rarely influencing the vehicle behaviour. In this paper, we address both limitations by introducing a system-level attack in which multiple dynamic elements (e.g., two pedestrians) carry adversarial patches (e.g., on cloths) and jointly amplify their effect through coordination and motion. We evaluate our attacks in the CARLA simulator using a state-of-the-art autonomous driving agent. At the system level, single-pedestrian attacks fail in all runs (out of 10), while dynamic collusion by two pedestrians induces full vehicle stops in up to 50\% of runs, with static collusion yielding no successful attack at all. These results show that system-level failures arise only when adversarial signals persist over time and are amplified through coordinated actors, exposing a gap between model-level robustness and end-to-end safety.