Progressive Multi-scale Light Field Networks

TL;DR

This paper introduces a progressive multi-scale light field network that enables efficient streaming, reduces rendering time, and mitigates aliasing by encoding multiple levels of detail with neural networks.

Contribution

The proposed network is the first to encode light fields at multiple levels of detail, supporting progressive streaming and anti-aliased rendering.

Findings

Supports progressive streaming with fewer neural network weights at lower levels

Reduces flickering and aliasing in high-resolution light field rendering

Enables per-pixel level of detail for foveated rendering

Abstract

Neural representations have shown great promise in their ability to represent radiance and light fields while being very compact compared to the image set representation. However, current representations are not well suited for streaming as decoding can only be done at a single level of detail and requires downloading the entire neural network model. Furthermore, high-resolution light field networks can exhibit flickering and aliasing as neural networks are sampled without appropriate filtering. To resolve these issues, we present a progressive multi-scale light field network that encodes a light field with multiple levels of detail. Lower levels of detail are encoded using fewer neural network weights enabling progressive streaming and reducing rendering time. Our progressive multi-scale light field network addresses aliasing by encoding smaller anti-aliased representations at its lower levels of detail. Additionally, per-pixel level of detail enables our representation to support dithered transitions and foveated rendering.