I$^2$-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs

TL;DR

I$^2$-SDF introduces a neural SDF-based framework utilizing differentiable raytracing for detailed indoor scene reconstruction, relighting, and editing, achieving superior results in quality and realism.

Contribution

The paper presents a novel neural SDF framework with a bubble loss and adaptive sampling for improved indoor scene reconstruction and a decomposition method for photorealistic relighting and editing.

Findings

Superior reconstruction quality on large-scale indoor scenes

Effective scene relighting and editing with realistic results

Outperforms state-of-the-art methods in view synthesis and scene editing

Abstract

In this work, we present I$^2$-SDF, a new method for intrinsic indoor scene reconstruction and editing using differentiable Monte Carlo raytracing on neural signed distance fields (SDFs). Our holistic neural SDF-based framework jointly recovers the underlying shapes, incident radiance and materials from multi-view images. We introduce a novel bubble loss for fine-grained small objects and error-guided adaptive sampling scheme to largely improve the reconstruction quality on large-scale indoor scenes. Further, we propose to decompose the neural radiance field into spatially-varying material of the scene as a neural field through surface-based, differentiable Monte Carlo raytracing and emitter semantic segmentations, which enables physically based and photorealistic scene relighting and editing applications. Through a number of qualitative and quantitative experiments, we demonstrate the superior quality of our method on indoor scene reconstruction, novel view synthesis, and scene editing compared to state-of-the-art baselines.