Geometry-Aware Global Feature Aggregation for Real-Time Indirect Illumination

TL;DR

This paper introduces a geometry-aware neural network architecture for real-time indirect illumination estimation in rendering, effectively capturing complex lighting, distant interreflections, and generalizing to new scenes, improving visual realism.

Contribution

It proposes a novel geometry-guided attention-based neural network for predicting global illumination in real-time rendering, addressing long-range indirect light and scene generalization.

Findings

Outperforms previous learning-based methods in complex lighting scenarios

Successfully captures distant indirect illumination and interreflections

Handles new unseen scenes effectively

Abstract

Real-time rendering with global illumination is crucial to afford the user realistic experience in virtual environments. We present a learning-based estimator to predict diffuse indirect illumination in screen space, which then is combined with direct illumination to synthesize globally-illuminated high dynamic range (HDR) results. Our approach tackles the challenges of capturing long-range/long-distance indirect illumination when employing neural networks and is generalized to handle complex lighting and scenarios. From the neural network thinking of the solver to the rendering equation, we present a novel network architecture to predict indirect illumination. Our network is equipped with a modified attention mechanism that aggregates global information guided by spacial geometry features, as well as a monochromatic design that encodes each color channel individually. We conducted extensive evaluations, and the experimental results demonstrate our superiority over previous learning-based techniques. Our approach excels at handling complex lighting such as varying-colored lighting and environment lighting. It can successfully capture distant indirect illumination and simulates the interreflections between textured surfaces well (i.e., color bleeding effects); it can also effectively handle new scenes that are not present in the training dataset.