Deep Appearance Prefiltering

TL;DR

This paper introduces a comprehensive multi-scale prefiltering framework for complex scene rendering that maintains appearance fidelity while significantly reducing memory and computational costs, independent of scene complexity.

Contribution

It presents the first multi-scale LoD method for prefiltering complex 3D environments using a neural representation that encodes appearance information efficiently.

Findings

Outperforms state-of-the-art prefiltering methods

Achieves significant memory savings for complex scenes

Maintains high appearance fidelity in rendering

Abstract

Physically based rendering of complex scenes can be prohibitively costly with a potentially unbounded and uneven distribution of complexity across the rendered image. The goal of an ideal level of detail (LoD) method is to make rendering costs independent of the 3D scene complexity, while preserving the appearance of the scene. However, current prefiltering LoD methods are limited in the appearances they can support due to their reliance of approximate models and other heuristics. We propose the first comprehensive multi-scale LoD framework for prefiltering 3D environments with complex geometry and materials (e.g., the Disney BRDF), while maintaining the appearance with respect to the ray-traced reference. Using a multi-scale hierarchy of the scene, we perform a data-driven prefiltering step to obtain an appearance phase function and directional coverage mask at each scale. At the heart of our approach is a novel neural representation that encodes this information into a compact latent form that is easy to decode inside a physically based renderer. Once a scene is baked out, our method requires no original geometry, materials, or textures at render time. We demonstrate that our approach compares favorably to state-of-the-art prefiltering methods and achieves considerable savings in memory for complex scenes.