Neural Voxel Renderer: Learning an Accurate and Controllable Rendering Tool

TL;DR

This paper introduces a neural voxel rendering framework capable of producing high-quality, controllable images from voxelized scenes, effectively handling textures, interactions, and modifications.

Contribution

It presents a novel neural rendering method that accurately maps voxelized scenes to images with controllable edits, outperforming existing approaches in detail and realism.

Findings

Produces highly detailed, realistic images from voxel inputs

Supports precise control over scene modifications and appearance changes

Handles low-resolution voxel grids effectively

Abstract

We present a neural rendering framework that maps a voxelized scene into a high quality image. Highly-textured objects and scene element interactions are realistically rendered by our method, despite having a rough representation as an input. Moreover, our approach allows controllable rendering: geometric and appearance modifications in the input are accurately propagated to the output. The user can move, rotate and scale an object, change its appearance and texture or modify the position of the light and all these edits are represented in the final rendering. We demonstrate the effectiveness of our approach by rendering scenes with varying appearance, from single color per object to complex, high-frequency textures. We show that our rerendering network can generate very detailed images that represent precisely the appearance of the input scene. Our experiments illustrate that our approach achieves more accurate image synthesis results compared to alternatives and can also handle low voxel grid resolutions. Finally, we show how our neural rendering framework can capture and faithfully render objects from real images and from a diverse set of classes.