ROI-NeRFs: Hi-Fi Visualization of Objects of Interest within a Scene by NeRFs Composition

TL;DR

This paper introduces ROI-NeRFs, a method for high-fidelity visualization of specific objects within large scenes by combining scene-wide and object-focused NeRFs, enhancing detail and efficiency.

Contribution

The study presents a novel framework that decomposes scenes into a global scene NeRF and multiple object-specific ROI NeRFs, enabling detailed visualization of objects of interest with efficient computation.

Findings

Improved level of detail for objects of interest in complex scenes.

Reduced artifacts and maintained efficiency in rendering.

Validated on real-world datasets, including cultural heritage environments.

Abstract

Efficient and accurate 3D reconstruction is essential for applications in cultural heritage. This study addresses the challenge of visualizing objects within large-scale scenes at a high level of detail (LOD) using Neural Radiance Fields (NeRFs). The aim is to improve the visual fidelity of chosen objects while maintaining the efficiency of the computations by focusing on details only for relevant content. The proposed ROI-NeRFs framework divides the scene into a Scene NeRF, which represents the overall scene at moderate detail, and multiple ROI NeRFs that focus on user-defined objects of interest. An object-focused camera selection module automatically groups relevant cameras for each NeRF training during the decomposition phase. In the composition phase, a Ray-level Compositional Rendering technique combines information from the Scene NeRF and ROI NeRFs, allowing simultaneous multi-object rendering composition. Quantitative and qualitative experiments conducted on two real-world datasets, including one on a complex eighteen's century cultural heritage room, demonstrate superior performance compared to baseline methods, improving LOD for object regions, minimizing artifacts, and without significantly increasing inference time.