R-PGA: Robust Physical Adversarial Camouflage Generation via Relightable 3D Gaussian Splatting

TL;DR

This paper introduces R-PGA, a novel framework for generating robust physical adversarial camouflage for autonomous vehicles, using relightable 3D Gaussian splatting to improve realism and robustness against environmental variations.

Contribution

The paper proposes a new attack framework leveraging relightable 3D Gaussian splatting and a hybrid rendering pipeline to enhance realism and robustness of physical adversarial camouflage.

Findings

Achieves photo-realistic reconstruction with decoupled material and lighting attributes.

Effectively mines worst-case configurations to improve robustness.

Flattens the loss landscape, ensuring consistent adversarial performance.

Abstract

Physical adversarial camouflage poses a severe security threat to autonomous driving systems by mapping adversarial textures onto 3D objects. Nevertheless, current methods remain brittle in complex dynamic scenarios, failing to generalize across diverse geometric (e.g., viewing configurations) and radiometric (e.g., dynamic illumination, atmospheric scattering) variations. We attribute this deficiency to two fundamental limitations in simulation and optimization. First, the reliance on coarse, oversimplified simulations (e.g., via CARLA) induces a significant domain gap, confining optimization to a biased feature space. Second, standard strategies targeting average performance result in a rugged loss landscape, leaving the camouflage vulnerable to configuration shifts.To bridge these gaps, we propose the Relightable Physical 3D Gaussian Splatting (3DGS) based Attack framework (R-PGA). Technically, to address the simulation fidelity issue, we leverage 3DGS to ensure photo-realistic reconstruction and augment it with physically disentangled attributes to decouple intrinsic material from lighting. Furthermore, we design a hybrid rendering pipeline that leverages precise Relightable 3DGS for foreground rendering, while employing a pre-trained image translation model to synthesize plausible relighted backgrounds that align with the relighted foreground.To address the optimization robustness issue, we propose the Hard Physical Configuration Mining (HPCM) module, designed to actively mine worst-case physical configurations and suppress their corresponding loss peaks. This strategy not only diminishes the overall loss magnitude but also effectively flattens the rugged loss landscape, ensuring consistent adversarial effectiveness and robustness across varying physical configurations.