A Convolutional Neural Deferred Shader for Physics Based Rendering

TL;DR

This paper introduces pbnds+, a physics-based neural deferred shading pipeline using convolutional neural networks to enhance shading and relighting, reducing parameters and improving performance over existing methods.

Contribution

It presents a novel neural rendering approach that employs CNNs and energy regularization to improve efficiency and accuracy in physics-based shading and relighting.

Findings

Outperforms classical baselines in shading tasks

Surpasses state-of-the-art neural shading models

Achieves better results than diffusion-based methods

Abstract

Recent advances in neural rendering have achieved impressive results on photorealistic shading and relighting, by using a multilayer perceptron (MLP) as a regression model to learn the rendering equation from a real-world dataset. Such methods show promise for photorealistically relighting real-world objects, which is difficult to classical rendering, as there is no easy-obtained material ground truth. However, significant challenges still remain the dense connections in MLPs result in a large number of parameters, which requires high computation resources, complicating the training, and reducing performance during rendering. Data driven approaches require large amounts of training data for generalization; unbalanced data might bias the model to ignore the unusual illumination conditions, e.g. dark scenes. This paper introduces pbnds+: a novel physics-based neural deferred shading pipeline utilizing convolution neural networks to decrease the parameters and improve the performance in shading and relighting tasks; Energy regularization is also proposed to restrict the model reflection during dark illumination. Extensive experiments demonstrate that our approach outperforms classical baselines, a state-of-the-art neural shading model, and a diffusion-based method.