MeshAdv: Adversarial Meshes for Visual Recognition

TL;DR

This paper introduces MeshAdv, a method for creating adversarial 3D meshes that can fool visual recognition models by manipulating shape features with minimal texture changes, using differentiable rendering for gradient computation.

Contribution

It presents a novel approach to generate adversarial 3D meshes that are physically plausible and effective against classifiers and detectors, extending adversarial attacks to 3D objects.

Findings

Adversarial meshes successfully attack classifiers and detectors.

Effective across different viewpoints and in black-box settings.

Minimal texture variation enhances physical realism.

Abstract

Highly expressive models such as deep neural networks (DNNs) have been widely applied to various applications. However, recent studies show that DNNs are vulnerable to adversarial examples, which are carefully crafted inputs aiming to mislead the predictions. Currently, the majority of these studies have focused on perturbation added to image pixels, while such manipulation is not physically realistic. Some works have tried to overcome this limitation by attaching printable 2D patches or painting patterns onto surfaces, but can be potentially defended because 3D shape features are intact. In this paper, we propose meshAdv to generate "adversarial 3D meshes" from objects that have rich shape features but minimal textural variation. To manipulate the shape or texture of the objects, we make use of a differentiable renderer to compute accurate shading on the shape and propagate the gradient. Extensive experiments show that the generated 3D meshes are effective in attacking both classifiers and object detectors. We evaluate the attack under different viewpoints. In addition, we design a pipeline to perform black-box attack on a photorealistic renderer with unknown rendering parameters.