PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems

TL;DR

This paper introduces PoInit-of-View, a method to perform transferable poisoning attacks on the SfM initialization in 3D reconstruction, disrupting key processes and degrading output quality across systems.

Contribution

It uncovers a new vulnerability in SfM initialization and proposes an adversarial perturbation method that transfers across multiple 3D reconstruction systems.

Findings

PoInit-of-View achieves a 25.1% increase in PSNR over baselines.

The method disrupts keypoint detection and feature matching.

It effectively transfers attacks from 3DGS to NeRF in black-box settings.

Abstract

Poisoning input views of 3D reconstruction systems has been recently studied. However, we identify that existing studies simply backpropagate adversarial gradients through the 3D reconstruction pipeline as a whole, without uncovering the new vulnerability rooted in specific modules of the 3D reconstruction pipeline. In this paper, we argue that the structure-from-motion (SfM) initialization, as the geometric core of many widely used reconstruction systems, can be targeted to achieve transferable poisoning effects across diverse 3D reconstruction systems. To this end, we propose PoInit-of-View, which optimizes adversarial perturbations to intentionally introduce cross-view gradient inconsistencies at projections of corresponding 3D points. These inconsistencies disrupt keypoint detection and feature matching, thereby corrupting pose estimation and triangulation within SfM, eventually resulting in low-quality rendered views. We also provide a theoretical analysis that connects cross-view inconsistency to correspondence collapse. Experimental results demonstrate the effectiveness of our PoInit-of-View on diverse 3D reconstruction systems and datasets, surpassing the single-view baseline by 25.1% in PSNR and 16.5% in SSIM in black-box transfer settings, such as 3DGS to NeRF.