AdvDO: Realistic Adversarial Attacks for Trajectory Prediction

TL;DR

This paper investigates the adversarial robustness of trajectory prediction models for autonomous vehicles, introducing a realistic attack method that significantly increases prediction errors and can cause unsafe driving behaviors in simulation.

Contribution

It presents a novel optimization-based adversarial attack framework for trajectory prediction and demonstrates its effectiveness and mitigation strategies.

Findings

Attack increases prediction error by over 50% and 37% on key metrics.

Adversarial trajectories can cause AVs to go off-road or collide in simulation.

Proposes an adversarial training scheme to improve robustness.

Abstract

Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors. While many prior works aim to achieve higher prediction accuracy, few study the adversarial robustness of their methods. To bridge this gap, we propose to study the adversarial robustness of data-driven trajectory prediction systems. We devise an optimization-based adversarial attack framework that leverages a carefully-designed differentiable dynamic model to generate realistic adversarial trajectories. Empirically, we benchmark the adversarial robustness of state-of-the-art prediction models and show that our attack increases the prediction error for both general metrics and planning-aware metrics by more than 50% and 37%. We also show that our attack can lead an AV to drive off road or collide into other vehicles in simulation. Finally, we demonstrate how to mitigate the adversarial attacks using an adversarial training scheme.