KiloNeuS: A Versatile Neural Implicit Surface Representation for Real-Time Rendering

TL;DR

KiloNeuS introduces a neural implicit surface representation using multiple tiny MLPs for real-time rendering, achieving high-quality geometry and normals, and enabling integration with path-tracing for global illumination.

Contribution

It presents a novel partitioned MLP approach for neural implicit surfaces, enabling real-time rendering and integration with path-tracing, addressing previous limitations of NeRF-based methods.

Findings

Achieves 46 FPS rendering on GPU with high resolution.

Outperforms single-MLP models in surface quality.

Successfully integrates with path-tracing for global illumination.

Abstract

NeRF-based techniques fit wide and deep multi-layer perceptrons (MLPs) to a continuous radiance field that can be rendered from any unseen viewpoint. However, the lack of surface and normals definition and high rendering times limit their usage in typical computer graphics applications. Such limitations have recently been overcome separately, but solving them together remains an open problem. We present KiloNeuS, a neural representation reconstructing an implicit surface represented as a signed distance function (SDF) from multi-view images and enabling real-time rendering by partitioning the space into thousands of tiny MLPs fast to inference. As we learn the implicit surface locally using independent models, resulting in a globally coherent geometry is non-trivial and needs to be addressed during training. We evaluate rendering performance on a GPU-accelerated ray-caster with in-shader neural network inference, resulting in an average of 46 FPS at high resolution, proving a satisfying tradeoff between storage costs and rendering quality. In fact, our evaluation for rendering quality and surface recovery shows that KiloNeuS outperforms its single-MLP counterpart. Finally, to exhibit the versatility of KiloNeuS, we integrate it into an interactive path-tracer taking full advantage of its surface normals. We consider our work a crucial first step toward real-time rendering of implicit neural representations under global illumination.