DivAS: Interactive 3D Segmentation of NeRFs via Depth-Weighted Voxel Aggregation

TL;DR

DivAS is an interactive, optimization-free framework for 3D segmentation of NeRFs that uses depth priors and fast GPU aggregation to enable real-time editing and high-quality segmentation without scene-specific training.

Contribution

We propose DivAS, a novel interactive segmentation method for NeRFs that combines user prompts, depth priors, and fast GPU aggregation, eliminating the need for scene-specific optimization.

Findings

Achieves segmentation quality comparable to optimization-based methods.

Runs 2-2.5x faster end-to-end compared to existing approaches.

Provides real-time visual feedback with a CUDA kernel under 200ms.

Abstract

Existing methods for segmenting Neural Radiance Fields (NeRFs) are often optimization-based, requiring slow per-scene training that sacrifices the zero-shot capabilities of 2D foundation models. We introduce DivAS (Depth-interactive Voxel Aggregation Segmentation), an optimization-free, fully interactive framework that addresses these limitations. Our method operates via a fast GUI-based workflow where 2D SAM masks, generated from user point prompts, are refined using NeRF-derived depth priors to improve geometric accuracy and foreground-background separation. The core of our contribution is a custom CUDA kernel that aggregates these refined multi-view masks into a unified 3D voxel grid in under 200ms, enabling real-time visual feedback. This optimization-free design eliminates the need for per-scene training. Experiments on Mip-NeRF 360{\deg} and LLFF show that DivAS achieves segmentation quality comparable to optimization-based methods, while being 2-2.5x faster end-to-end, and up to an order of magnitude faster when excluding user prompting time.