Real-Time Rendering of Arbitrary Surface Geometries using Learnt Transfer

TL;DR

This paper introduces a learned transfer method using an MLP to enable real-time photorealistic rendering of arbitrary surface geometries, especially useful for implicit surfaces without traditional mesh or UV mappings.

Contribution

The paper presents a novel compact transfer representation learned directly on scene geometry points, bypassing the need for dense meshes or UV maps, suitable for implicit surfaces.

Findings

Enables real-time rendering of diffuse and glossy materials on SDF geometries.

Achieves photorealistic effects without dense mesh tessellation or UV mapping.

Demonstrates effectiveness on arbitrary surface geometries.

Abstract

Precomputed Radiance Transfer (PRT) is widely used for real-time photorealistic effects. PRT disentangles the rendering equation into transfer and lighting, enabling their precomputation. Transfer accounts for the cosine-weighted visibility of points in the scene while lighting for emitted radiance from the environment. Prior art stored precomputed transfer in a tabulated manner, either in vertex or texture space. These values are fetched with interpolation at each point for shading. Vertex space methods require densely tessellated mesh vertices for high quality images. Texture space methods require non-overlapping and area-preserving UV mapping to be available. They also require a high-resolution texture to avoid rendering artifacts. In this paper, we propose a compact transfer representation that is learnt directly on scene geometry points. Specifically, we train a small multi-layer perceptron (MLP) to predict the transfer at sampled surface points. Our approach is most beneficial where inherent mesh storage structure and natural UV mapping are not available, such as Implicit Surfaces as it learns the transfer values directly on the surface. We demonstrate real-time, photorealistic renderings of diffuse and glossy materials on SDF geometries with PRT using our approach.