Manipulating Trajectory Prediction with Backdoors

TL;DR

This paper explores the vulnerability of trajectory prediction models in autonomous vehicles to backdoor attacks, demonstrating how specific triggers can manipulate predictions and evaluating potential defenses against such security threats.

Contribution

It introduces the concept of backdoors in trajectory prediction, identifies triggers that can manipulate model outputs, and evaluates defense strategies to mitigate these security risks.

Findings

Backdoors can be triggered by specific maneuvers like braking or curves.

Models remain accurate in benign conditions but are vulnerable to manipulated inputs.

Clustering shows promise as a defense mechanism against backdoors.

Abstract

Autonomous vehicles ought to predict the surrounding agents' trajectories to allow safe maneuvers in uncertain and complex traffic situations. As companies increasingly apply trajectory prediction in the real world, security becomes a relevant concern. In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction. To this end, we describe and investigate four triggers that could affect trajectory prediction. We then show that these triggers (for example, a braking vehicle), when correlated with a desired output (for example, a curve) during training, cause the desired output of a state-of-the-art trajectory prediction model. In other words, the model has good benign performance but is vulnerable to backdoors. This is the case even if the trigger maneuver is performed by a non-casual agent behind the target vehicle. As a side-effect, our analysis reveals interesting limitations within trajectory prediction models. Finally, we evaluate a range of defenses against backdoors. While some, like simple offroad checks, do not enable detection for all triggers, clustering is a promising candidate to support manual inspection to find backdoors.