A Generalizable Light Transport 3D Embedding for Global Illumination

TL;DR

This paper introduces a neural 3D embedding for global illumination that generalizes across diverse scenes, enabling efficient and adaptable rendering without explicit ray tracing cues.

Contribution

It proposes a scalable transformer-based 3D scene embedding that models global light transport directly from scene geometry and materials, improving cross-scene generalization.

Findings

Effective diffuse GI prediction across various indoor scenes.

Embedding can be quickly fine-tuned for different rendering tasks.

Preliminary results show potential for radiance field estimation and path guiding.

Abstract

Global illumination (GI) is essential for realistic rendering but remains computationally expensive due to the complexity of simulating indirect light transport. Recent neural methods have mainly relied on per-scene optimization, sometimes extended to handle changes in camera or geometry. Efforts toward cross-scene generalization have largely stayed in 2D screen space, such as neural denoising or G-buffer based GI prediction, which often suffer from view inconsistency and limited spatial understanding. We propose a generalizable 3D light transport embedding that approximates global illumination directly from 3D scene configurations, without using rasterized or path-traced cues. Each scene is represented as a point cloud with geometric and material features. A scalable transformer models global point-to-point interactions to encode these features into neural primitives. At render time, each query point retrieves nearby primitives via nearest-neighbor search and aggregates their latent features through cross-attention to predict the desired rendering quantity. We demonstrate results on diffuse global illumination prediction across diverse indoor scenes with varying layouts, geometry, and materials. The embedding trained for irradiance estimation can be quickly adapted to new rendering tasks with limited fine-tuning. We also present preliminary results for spatial-directional radiance field estimation for glossy materials and show how the normalized field can accelerate unbiased path guiding. This approach highlights a path toward integrating learned priors into rendering pipelines without explicit ray-traced illumination cues.