Real-time Neural Six-way Lightmaps

TL;DR

This paper introduces a neural method for generating six-way lightmaps in real-time, enabling realistic and interactive smoke rendering in virtual environments with efficient computation.

Contribution

It proposes a neural approach that predicts six-way lightmaps from guiding maps, balancing realism and efficiency for real-time smoke rendering in games and VR/AR.

Findings

Supports real-time user interaction including camera movement and light changes

Achieves a good balance between visual realism and computational efficiency

Demonstrates suitability for game and VR/AR applications

Abstract

Participating media are a pervasive and intriguing visual effect in virtual environments. Unfortunately, rendering such phenomena in real-time is notoriously difficult due to the computational expense of estimating the volume rendering equation. While the six-way lightmaps technique has been widely used in video games to render smoke with a camera-oriented billboard and approximate lighting effects using six precomputed lightmaps, achieving a balance between realism and efficiency, it is limited to pre-simulated animation sequences and is ignorant of camera movement. In this work, we propose a neural six-way lightmaps method to strike a long-sought balance between dynamics and visual realism. Our approach first generates a guiding map from the camera view using ray marching with a large sampling distance to approximate smoke scattering and silhouette. Then, given a guiding map, we train a neural network to predict the corresponding six-way lightmaps. The resulting lightmaps can be seamlessly used in existing game engine pipelines. This approach supports visually appealing rendering effects while enabling real-time user interactivity, including smoke-obstacle interaction, camera movement, and light change. By conducting a series of comprehensive benchmarks, we demonstrate that our method is well-suited for real-time applications, such as games and VR/AR.