Enhancing Real-World Adversarial Patches through 3D Modeling of Complex Target Scenes

TL;DR

This paper introduces a 3D modeling framework to craft and evaluate adversarial patches in realistic environments, improving their robustness and reproducibility in real-world scenarios.

Contribution

The study presents a novel 3D modeling approach for creating and testing adversarial patches, addressing limitations of previous 2D-based methods and enhancing real-world applicability.

Findings

Adversarial patches generated are robust across various real-world conditions.

The framework enables reproducible evaluation in digital and physical environments.

Improved attack effectiveness demonstrated in complex 3D scenes.

Abstract

Adversarial examples have proven to be a concerning threat to deep learning models, particularly in the image domain. However, while many studies have examined adversarial examples in the real world, most of them relied on 2D photos of the attack scene. As a result, the attacks proposed may have limited effectiveness when implemented in realistic environments with 3D objects or varied conditions. There are few studies on adversarial learning that use 3D objects, and in many cases, other researchers are unable to replicate the real-world evaluation process. In this study, we present a framework that uses 3D modeling to craft adversarial patches for an existing real-world scene. Our approach uses a 3D digital approximation of the scene as a simulation of the real world. With the ability to add and manipulate any element in the digital scene, our framework enables the attacker to improve the adversarial patch's impact in real-world settings. We use the framework to create a patch for an everyday scene and evaluate its performance using a novel evaluation process that ensures that our results are reproducible in both the digital space and the real world. Our evaluation results show that the framework can generate adversarial patches that are robust to different settings in the real world.