On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles

TL;DR

This paper investigates the adversarial robustness of trajectory prediction models for autonomous vehicles, revealing vulnerabilities where small trajectory perturbations can cause significant prediction errors and unsafe driving decisions.

Contribution

It introduces a new adversarial attack method for trajectory prediction and evaluates its impact across multiple models and datasets, highlighting safety concerns.

Findings

Adversarial prediction error increases by over 150%.

Adversarial trajectories can lead to unsafe vehicle behaviors.

Mitigation techniques like data augmentation can reduce vulnerability.

Abstract

Trajectory prediction is a critical component for autonomous vehicles (AVs) to perform safe planning and navigation. However, few studies have analyzed the adversarial robustness of trajectory prediction or investigated whether the worst-case prediction can still lead to safe planning. To bridge this gap, we study the adversarial robustness of trajectory prediction models by proposing a new adversarial attack that perturbs normal vehicle trajectories to maximize the prediction error. Our experiments on three models and three datasets show that the adversarial prediction increases the prediction error by more than 150%. Our case studies show that if an adversary drives a vehicle close to the target AV following the adversarial trajectory, the AV may make an inaccurate prediction and even make unsafe driving decisions. We also explore possible mitigation techniques via data augmentation and trajectory smoothing. The implementation is open source at https://github.com/zqzqz/AdvTrajectoryPrediction.