Exploring the Physical World Adversarial Robustness of Vehicle Detection

TL;DR

This paper introduces a new dataset and simulation pipeline to evaluate vehicle detection models' robustness against physical adversarial attacks in real-world-like scenarios, revealing model resilience and attack effectiveness.

Contribution

The study presents the DCI dataset and a simulation pipeline for standardized evaluation of adversarial robustness in vehicle detection models.

Findings

Yolo v6 shows only 6.59% AP drop under attack

ASA attack causes 14.51% AP reduction, twice other algorithms

Static scenes and weather variations have minimal impact on detection performance

Abstract

Adversarial attacks can compromise the robustness of real-world detection models. However, evaluating these models under real-world conditions poses challenges due to resource-intensive experiments. Virtual simulations offer an alternative, but the absence of standardized benchmarks hampers progress. Addressing this, we propose an innovative instant-level data generation pipeline using the CARLA simulator. Through this pipeline, we establish the Discrete and Continuous Instant-level (DCI) dataset, enabling comprehensive experiments involving three detection models and three physical adversarial attacks. Our findings highlight diverse model performances under adversarial conditions. Yolo v6 demonstrates remarkable resilience, experiencing just a marginal 6.59% average drop in average precision (AP). In contrast, the ASA attack yields a substantial 14.51% average AP reduction, twice the effect of other algorithms. We also note that static scenes yield higher recognition AP values, and outcomes remain relatively consistent across varying weather conditions. Intriguingly, our study suggests that advancements in adversarial attack algorithms may be approaching its ``limitation''.In summary, our work underscores the significance of adversarial attacks in real-world contexts and introduces the DCI dataset as a versatile benchmark. Our findings provide valuable insights for enhancing the robustness of detection models and offer guidance for future research endeavors in the realm of adversarial attacks.