Bait and Switch: Online Training Data Poisoning of Autonomous Driving Systems

TL;DR

This paper demonstrates how an adversary can subtly poison the online training data of autonomous driving perception systems by controlling environmental factors, leading to degraded performance and potential system malfunction.

Contribution

It introduces a novel physical environment-based data poisoning attack method applicable during online fine-tuning of autonomous vehicle perception models.

Findings

Environmental perturbations can induce spurious concepts in DNNs during online training.

Poisoned models show significant accuracy degradation in real-world scenarios.

Attack method does not require modifying traffic lights or labels.

Abstract

We show that by controlling parts of a physical environment in which a pre-trained deep neural network (DNN) is being fine-tuned online, an adversary can launch subtle data poisoning attacks that degrade the performance of the system. While the attack can be applied in general to any perception task, we consider a DNN based traffic light classifier for an autonomous car that has been trained in one city and is being fine-tuned online in another city. We show that by injecting environmental perturbations that do not modify the traffic lights themselves or ground-truth labels, the adversary can cause the deep network to learn spurious concepts during the online learning phase. The attacker can leverage the introduced spurious concepts in the environment to cause the model's accuracy to degrade during operation; therefore, causing the system to malfunction.