Learning Transferable 3D Adversarial Cloaks for Deep Trained Detectors

TL;DR

This paper introduces a 3D adversarial patch attack that is trained on human meshes and can fool deep object detectors across different poses and viewpoints, demonstrating robustness in real-world scenarios.

Contribution

The paper proposes a novel 3D adversarial patch training method using differentiable rendering, enabling persistent and robust attacks on deep detectors in physical environments.

Findings

Successfully fools state-of-the-art detectors under various poses

Demonstrates robustness of 3D patches in real-world conditions

Introduces a differentiable rendering pipeline for adversarial training

Abstract

This paper presents a novel patch-based adversarial attack pipeline that trains adversarial patches on 3D human meshes. We sample triangular faces on a reference human mesh, and create an adversarial texture atlas over those faces. The adversarial texture is transferred to human meshes in various poses, which are rendered onto a collection of real-world background images. Contrary to the traditional patch-based adversarial attacks, where prior work attempts to fool trained object detectors using appended adversarial patches, this new form of attack is mapped into the 3D object world and back-propagated to the texture atlas through differentiable rendering. As such, the adversarial patch is trained under deformation consistent with real-world materials. In addition, and unlike existing adversarial patches, our new 3D adversarial patch is shown to fool state-of-the-art deep object detectors robustly under varying views, potentially leading to an attacking scheme that is persistently strong in the physical world.