Real-Time Volume-Rendering Image Denoising Based on Spatiotemporal Weighted Kernel Prediction

TL;DR

This paper introduces a lightweight neural network for real-time image denoising in volumetric path tracing, improving quality and temporal stability with fewer samples.

Contribution

A spatiotemporal neural network that uses dual-input encoding and reprojection for better denoising in real-time volumetric rendering.

Findings

The proposed method outperforms baseline models in feature extraction and detail representation.

It effectively suppresses noise while maintaining temporal stability in rendered images.

The dual-input architecture improves performance with low samples per pixel.

Abstract

Volumetric Path Tracing (VPT) based on Monte Carlo (MC) sampling often requires numerous samples for high-quality images, but real-time applications limit samples to maintain interaction rates, leading to significant noise. Traditional real-time denoising methods use radiance and geometric features as neural network inputs, but lightweight networks struggle with temporal stability and complex mapping relationships, causing blurry results. To address these issues, a spatiotemporal lightweight neural network is proposed to enhance the denoising performance of VPT-rendered images with low samples per pixel. First, the reprojection technique was employed to obtain features from historical frames. Next, a dual-input convolutional neural network architecture was designed to predict filtering kernels. Radiance and geometric features were encoded independently. The encoding of geometric features guided the pixel-wise fitting of radiance feature filters. Finally, learned weight filtering kernels were applied to images’ spatiotemporal filtering to produce denoised results. The experimental results across multiple denoising datasets demonstrate that this approach outperformed the baseline models in terms of feature extraction and detail representation capabilities while effectively suppressing noise with superior performance and enhanced temporal stability.

Figures