Neural-GASh: A CGA-based neural radiance prediction pipeline for real-time shading

TL;DR

Neural-GASh introduces a real-time neural radiance prediction pipeline using CGA-encoded data for dynamic scene shading, eliminating the need for precomputations and enabling efficient rendering in interactive environments.

Contribution

The paper presents Neural-GASh, a novel neural radiance field approach that directly uses CGA-based vertex information for real-time shading without precomputations, integrated into Unity.

Findings

Achieves real-time shading for animated meshes in Unity.

Demonstrates competitive performance with traditional PRT methods.

Works effectively on scenes generated via 3D Gaussian splats.

Abstract

This paper presents Neural-GASh, a novel real-time shading pipeline for 3D meshes, that leverages a neural radiance field architecture to perform image-based rendering (IBR) using Conformal Geometric Algebra (CGA)-encoded vertex information as input. Unlike traditional Precomputed Radiance Transfer (PRT) methods, that require expensive offline precomputations, our learned model directly consumes CGA-based representations of vertex positions and normals, enabling dynamic scene shading without precomputation. Integrated seamlessly into the Unity engine, Neural-GASh facilitates accurate shading of animated and deformed 3D meshes - capabilities essential for dynamic, interactive environments. The shading of the scene is implemented within Unity, where rotation of scene lights in terms of Spherical Harmonics is also performed optimally using CGA. This neural field approach is designed to deliver fast and efficient light transport simulation across diverse platforms, including mobile and VR, while preserving high rendering quality. Additionally, we evaluate our method on scenes generated via 3D Gaussian splats, further demonstrating the flexibility and robustness of Neural-GASh in diverse scenarios. Performance is evaluated in comparison to conventional PRT, demonstrating competitive rendering speeds even with complex geometries.