DTA: Physical Camouflage Attacks using Differentiable Transformation Network

TL;DR

This paper introduces DTA, a novel framework that generates robust physical adversarial camouflage for 3D objects, effectively evading object detection models across various real-world transformations by leveraging a differentiable transformation network.

Contribution

The paper proposes the Differentiable Transformation Network (DTN) and a framework for creating physical adversarial camouflage that accounts for diverse transformations, improving robustness over previous neural renderer-based methods.

Findings

Camouflaged 3D vehicles evade state-of-the-art detection models in photo-realistic environments.

The method successfully transfers from virtual environments to real-world objects like a scaled Tesla Model 3.

DTA enhances the robustness of physical adversarial attacks against various transformations.

Abstract

To perform adversarial attacks in the physical world, many studies have proposed adversarial camouflage, a method to hide a target object by applying camouflage patterns on 3D object surfaces. For obtaining optimal physical adversarial camouflage, previous studies have utilized the so-called neural renderer, as it supports differentiability. However, existing neural renderers cannot fully represent various real-world transformations due to a lack of control of scene parameters compared to the legacy photo-realistic renderers. In this paper, we propose the Differentiable Transformation Attack (DTA), a framework for generating a robust physical adversarial pattern on a target object to camouflage it against object detection models with a wide range of transformations. It utilizes our novel Differentiable Transformation Network (DTN), which learns the expected transformation of a rendered object when the texture is changed while preserving the original properties of the target object. Using our attack framework, an adversary can gain both the advantages of the legacy photo-realistic renderers including various physical-world transformations and the benefit of white-box access by offering differentiability. Our experiments show that our camouflaged 3D vehicles can successfully evade state-of-the-art object detection models in the photo-realistic environment (i.e., CARLA on Unreal Engine). Furthermore, our demonstration on a scaled Tesla Model 3 proves the applicability and transferability of our method to the real world.