PruNeRF: Segment-Centric Dataset Pruning via 3D Spatial Consistency

TL;DR

PruNeRF is a novel dataset pruning framework that uses 3D spatial consistency and segmentation to effectively identify and remove distractors, improving NeRF robustness.

Contribution

It introduces a segment-centric pruning method leveraging 3D spatial consistency and segmentation, enhancing distractor removal in NeRF training datasets.

Findings

Outperforms state-of-the-art methods in distractor removal

Improves NeRF robustness against distractors

Utilizes influence functions and depth-based reprojection techniques

Abstract

Neural Radiance Fields (NeRF) have shown remarkable performance in learning 3D scenes. However, NeRF exhibits vulnerability when confronted with distractors in the training images -- unexpected objects are present only within specific views, such as moving entities like pedestrians or birds. Excluding distractors during dataset construction is a straightforward solution, but without prior knowledge of their types and quantities, it becomes prohibitively expensive. In this paper, we propose PruNeRF, a segment-centric dataset pruning framework via 3D spatial consistency, that effectively identifies and prunes the distractors. We first examine existing metrics for measuring pixel-wise distraction and introduce Influence Functions for more accurate measurements. Then, we assess 3D spatial consistency using a depth-based reprojection technique to obtain 3D-aware distraction. Furthermore, we incorporate segmentation for pixel-to-segment refinement, enabling more precise identification. Our experiments on benchmark datasets demonstrate that PruNeRF consistently outperforms state-of-the-art methods in robustness against distractors.