Physical Evaluation of Naturalistic Adversarial Patches for Camera-Based Traffic-Sign Detection

TL;DR

This study evaluates the physical transferability of Naturalistic Adversarial Patches in traffic sign detection for autonomous vehicles, demonstrating how different configurations impact detector confidence and highlighting the importance of systematic evaluation protocols.

Contribution

It introduces a customized dataset and physical evaluation protocols for assessing adversarial patches in AV traffic sign detection, advancing the understanding of real-world attack transferability.

Findings

NAPs reduce STOP class confidence in various configurations

The CompGTSRB dataset effectively facilitates physical adversarial patch evaluation

Systematic protocols improve the credibility of patch transferability assessments

Abstract

This paper studies how well Naturalistic Adversarial Patches (NAPs) transfer to a physical traffic sign setting when the detector is trained on a customized dataset for an autonomous vehicle (AV) environment. We construct a composite dataset, CompGTSRB (which is customized dataset for AV environment), by pasting traffic sign instances from the German Traffic Sign Recognition Benchmark (GTSRB) onto undistorted backgrounds captured from the target platform. CompGTSRB is used to train a YOLOv5 model and generate patches using a Generative Adversarial Network (GAN) with latent space optimization, following existing NAP methods. We carried out a series of experiments on our Quanser QCar testbed utilizing the front CSI camera provided in QCar. Across configurations, NAPs reduce the detector's STOP class confidence. Different configurations include distance, patch sizes, and patch placement. These results along with a detailed step-by-step methodology indicate the utility of CompGTSRB dataset and the proposed systematic physical protocols for credible patch evaluation. The research further motivate researching the defenses that address localized patch corruption in embedded perception pipelines.