Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering

TL;DR

Normal-GS introduces a novel method integrating normal vectors into 3D Gaussian Splatting, improving surface normal accuracy and rendering quality while maintaining real-time performance.

Contribution

It models the interaction between normals and lighting using a physically-based approach, re-parameterizes surface colors, and employs an anchor-based encoding to enhance normal estimation and rendering.

Findings

Achieves near state-of-the-art visual quality.

Provides accurate surface normal estimation.

Maintains real-time rendering speeds.

Abstract

Rendering and reconstruction are long-standing topics in computer vision and graphics. Achieving both high rendering quality and accurate geometry is a challenge. Recent advancements in 3D Gaussian Splatting (3DGS) have enabled high-fidelity novel view synthesis at real-time speeds. However, the noisy and discrete nature of 3D Gaussian primitives hinders accurate surface estimation. Previous attempts to regularize 3D Gaussian normals often degrade rendering quality due to the fundamental disconnect between normal vectors and the rendering pipeline in 3DGS-based methods. Therefore, we introduce Normal-GS, a novel approach that integrates normal vectors into the 3DGS rendering pipeline. The core idea is to model the interaction between normals and incident lighting using the physically-based rendering equation. Our approach re-parameterizes surface colors as the product of normals and a designed Integrated Directional Illumination Vector (IDIV). To optimize memory usage and simplify optimization, we employ an anchor-based 3DGS to implicitly encode locally-shared IDIVs. Additionally, Normal-GS leverages optimized normals and Integrated Directional Encoding (IDE) to accurately model specular effects, enhancing both rendering quality and surface normal precision. Extensive experiments demonstrate that Normal-GS achieves near state-of-the-art visual quality while obtaining accurate surface normals and preserving real-time rendering performance.