Neural Adaptive SCEne Tracing

TL;DR

NAScenT introduces a hybrid explicit-implicit neural scene representation with hierarchical sampling, significantly improving training efficiency and reconstruction quality for complex and large-scale scenes.

Contribution

It is the first neural rendering method to combine a hierarchical octree with a two-stage sampling process for faster training and better scene reconstruction.

Findings

Outperforms existing methods in training speed and quality

Effective on large outdoor and complex small scenes

Uses a hybrid explicit-implicit neural representation

Abstract

Neural rendering with implicit neural networks has recently emerged as an attractive proposition for scene reconstruction, achieving excellent quality albeit at high computational cost. While the most recent generation of such methods has made progress on the rendering (inference) times, very little progress has been made on improving the reconstruction (training) times. In this work, we present Neural Adaptive Scene Tracing (NAScenT), the first neural rendering method based on directly training a hybrid explicit-implicit neural representation. NAScenT uses a hierarchical octree representation with one neural network per leaf node and combines this representation with a two-stage sampling process that concentrates ray samples where they matter most near object surfaces. As a result, NAScenT is capable of reconstructing challenging scenes including both large, sparsely populated volumes like UAV captured outdoor environments, as well as small scenes with high geometric complexity. NAScenT outperforms existing neural rendering approaches in terms of both quality and training time.