AdvReal: Physical Adversarial Patch Generation Framework for Security Evaluation of Object Detection Systems

TL;DR

AdvReal introduces a unified framework for generating physical adversarial patches that effectively fool object detection systems in real-world scenarios, demonstrating high success rates and robustness across viewpoints and lighting conditions.

Contribution

The paper presents a novel joint adversarial training framework for 2D and 3D domains, incorporating realistic enhancement modules for physical-world adversarial patch generation.

Findings

Achieves 70.13% attack success rate on YOLOv12 in physical scenarios.

Maintains over 90% success rate across multiple viewpoints and distances.

Outperforms existing methods like T-SEA and AdvTexture significantly.

Abstract

Autonomous vehicles are typical complex intelligent systems with artificial intelligence at their core. However, perception methods based on deep learning are extremely vulnerable to adversarial samples, resulting in security accidents. How to generate effective adversarial examples in the physical world and evaluate object detection systems is a huge challenge. In this study, we propose a unified joint adversarial training framework for both 2D and 3D domains, which simultaneously optimizes texture maps in 2D image and 3D mesh spaces to better address intra-class diversity and real-world environmental variations. The framework includes a novel realistic enhanced adversarial module, with time-space and relighting mapping pipeline that adjusts illumination consistency between adversarial patches and target garments under varied viewpoints. Building upon this, we develop a realism enhancement mechanism that incorporates non-rigid deformation modeling and texture remapping to ensure alignment with the human body's non-rigid surfaces in 3D scenes. Extensive experiment results in digital and physical environments demonstrate that the adversarial textures generated by our method can effectively mislead the target detection model. Specifically, our method achieves an average attack success rate (ASR) of 70.13% on YOLOv12 in physical scenarios, significantly outperforming existing methods such as T-SEA (21.65%) and AdvTexture (19.70%). Moreover, the proposed method maintains stable ASR across multiple viewpoints and distances, with an average attack success rate exceeding 90% under both frontal and oblique views at a distance of 4 meters. This confirms the method's strong robustness and transferability under multi-angle attacks, varying lighting conditions, and real-world distances. The demo video and code can be obtained at https://github.com/Huangyh98/AdvReal.git.