Systematic Discovery of Semantic Attacks in Online Map Construction through Conditional Diffusion

TL;DR

This paper introduces MIRAGE, a diffusion model-based framework that systematically discovers semantic attacks on online map construction for autonomous vehicles, bypassing defenses and causing significant mapping errors.

Contribution

MIRAGE leverages diffusion models to find realistic semantic scene variations that deceive mapping systems, revealing vulnerabilities in current adversarial defenses.

Findings

MIRAGE suppresses 57.7% of detections and corrupts 96% of trajectories in boundary removal attacks.

MIRAGE successfully injects fictitious boundaries, unlike prior pixel-level attacks.

Semantic perturbations are more challenging to defend against than pixel-level attacks.

Abstract

Autonomous vehicles depend on online HD map construction to perceive lane boundaries, dividers, and pedestrian crossings -- safety-critical road elements that directly govern motion planning. While existing pixel perturbation attacks can disrupt the mapping, they can be neutralized by standard adversarial defenses. We present MIRAGE, a framework for systematic discovery of semantic attacks that bypass adversarial defenses and degrade mapping predictions by finding plausible environmental variation (e.g. shadows, wet roads). MIRAGE exploits the latent manifold of real-world data learned by diffusion models, and searches for semantically mutated scenes neighboring the ground truth with the same road topology yet mislead the mapping predictions. We evaluate MIRAGE on nuScenes and demonstrate two attacks: (1) boundary removal, suppressing 57.7% of detections and corrupting 96% of planned trajectories; and (2) boundary injection, the only method that successfully injects fictitious boundaries, while pixel PGD and AdvPatch fail entirely. Both attacks remain potent under various adversarial defenses. We use two independent VLM judges to quantify realism, where MIRAGE passes as realistic 80--84% of the time (vs. 97--99% for clean nuScenes), while AdvPatch only 0--9%. Our findings expose a categorical gap in current adversarial defenses: semantic-level perturbations that manifest as legitimate environmental variation are substantially harder to mitigate than pixel-level perturbations.