Gaussian Splatting Under Attack: Investigating Adversarial Noise in 3D Objects

TL;DR

This paper introduces a novel adversarial attack method, M-IFGSM, targeting 3D Gaussian Splatting models and CLIP, revealing significant vulnerabilities in 3D object detection with minimal perceptible noise.

Contribution

The work presents M-IFGSM, a new adversarial attack technique specifically designed for 3D models, demonstrating its effectiveness in degrading CLIP's zero-shot detection accuracy on 3D objects.

Findings

Adversarial noise reduces top-1 accuracy from 95.4% to 12.5% on train images.

The attack causes a significant drop in detection confidence levels.

Adversarial perturbations are nearly imperceptible to humans.

Abstract

3D Gaussian Splatting has advanced radiance field reconstruction, enabling high-quality view synthesis and fast rendering in 3D modeling. While adversarial attacks on object detection models are well-studied for 2D images, their impact on 3D models remains underexplored. This work introduces the Masked Iterative Fast Gradient Sign Method (M-IFGSM), designed to generate adversarial noise targeting the CLIP vision-language model. M-IFGSM specifically alters the object of interest by focusing perturbations on masked regions, degrading the performance of CLIP's zero-shot object detection capability when applied to 3D models. Using eight objects from the Common Objects 3D (CO3D) dataset, we demonstrate that our method effectively reduces the accuracy and confidence of the model, with adversarial noise being nearly imperceptible to human observers. The top-1 accuracy in original model renders drops from 95.4\% to 12.5\% for train images and from 91.2\% to 35.4\% for test images, with confidence levels reflecting this shift from true classification to misclassification, underscoring the risks of adversarial attacks on 3D models in applications such as autonomous driving, robotics, and surveillance. The significance of this research lies in its potential to expose vulnerabilities in modern 3D vision models, including radiance fields, prompting the development of more robust defenses and security measures in critical real-world applications.