RenderBender: A Survey on Adversarial Attacks Using Differentiable Rendering

TL;DR

This survey reviews how differentiable rendering techniques enable physically plausible adversarial attacks on neural networks, unifying diverse goals and identifying future research directions in the field.

Contribution

It introduces the first framework that unifies various adversarial attack goals and tasks using differentiable rendering, facilitating comparison and highlighting research gaps.

Findings

Unified framework for diverse attack goals

Identification of research gaps in adversarial attacks

Future directions for real-world threat studies

Abstract

Differentiable rendering techniques like Gaussian Splatting and Neural Radiance Fields have become powerful tools for generating high-fidelity models of 3D objects and scenes. Their ability to produce both physically plausible and differentiable models of scenes are key ingredient needed to produce physically plausible adversarial attacks on DNNs. However, the adversarial machine learning community has yet to fully explore these capabilities, partly due to differing attack goals (e.g., misclassification, misdetection) and a wide range of possible scene manipulations used to achieve them (e.g., alter texture, mesh). This survey contributes the first framework that unifies diverse goals and tasks, facilitating easy comparison of existing work, identifying research gaps, and highlighting future directions - ranging from expanding attack goals and tasks to account for new modalities, state-of-the-art models, tools, and pipelines, to underscoring the importance of studying real-world threats in complex scenes.