Neural Dynamic GI: Random-Access Neural Compression for Temporal Lightmaps in Dynamic Lighting Environments

TL;DR

Neural Dynamic GI introduces a neural compression method for temporal lightmaps that significantly reduces storage needs while maintaining high-quality real-time global illumination in dynamic environments.

Contribution

The paper presents a novel neural compression technique for temporal lightmaps, integrating multi-dimensional features and lightweight neural networks for efficient real-time decompression.

Findings

Achieves high-quality dynamic GI with low storage and memory requirements.

Uses block compression simulation during training to enhance compression ratio.

Enables real-time decompression with modest overhead.

Abstract

High-quality global illumination (GI) in real-time rendering is commonly achieved using precomputed lighting techniques, with lightmap as the standard choice. To support GI for static objects in dynamic lighting environments, multiple lightmaps at different lighting conditions need to be precomputed, which incurs substantial storage and memory overhead. To overcome this limitation, we propose Neural Dynamic GI (NDGI), a novel compression technique specifically designed for temporal lightmap sets. Our method utilizes multi-dimensional feature maps and lightweight neural networks to integrate the temporal information instead of storing multiple sets explicitly, which significantly reduces the storage size of lightmaps. Additionally, we introduce a block compression (BC) simulation strategy during the training process, which enables BC compression on the final generated feature maps and further improves the compression ratio. To enable efficient real-time decompression, we also integrate a virtual texturing (VT) system with our neural representation. Compared with prior methods, our approach achieves high-quality dynamic GI while maintaining remarkably low storage and memory requirements, with only modest real-time decompression overhead. To facilitate further research in this direction, we will release our temporal lightmap dataset precomputed in multiple scenes featuring diverse temporal variations.