A 3DGS-Diffusion Self-Supervised Framework for Normal Estimation from a Single Image

TL;DR

This paper introduces SINGAD, a self-supervised framework that leverages physics-based light interaction modeling and differentiable rendering to improve normal estimation from a single image, overcoming multi-view inconsistency and data annotation issues.

Contribution

It proposes a novel 3D Gaussian splatting diffusion model with physics-driven light interaction and a differentiable reprojection loss for self-supervised normal estimation from a single image.

Findings

Outperforms state-of-the-art methods on Google Scanned Objects dataset.

Effectively enforces multi-view normal consistency.

Eliminates the need for dense normal annotations.

Abstract

The lack of spatial dimensional information remains a challenge in normal estimation from a single image. Recent diffusion-based methods have demonstrated significant potential in 2D-to-3D implicit mapping, they rely on data-driven statistical priors and miss the explicit modeling of light-surface interaction, leading to multi-view normal direction conflicts. Moreover, the discrete sampling mechanism of diffusion models causes gradient discontinuity in differentiable rendering reconstruction modules, preventing 3D geometric errors from being backpropagated to the normal generation network, thereby forcing existing methods to depend on dense normal annotations. This paper proposes SINGAD, a novel Self-supervised framework from a single Image for Normal estimation via 3D GAussian splatting guided Diffusion. By integrating physics-driven light-interaction modeling and a differentiable rendering-based reprojection strategy, our framework directly converts 3D geometric errors into normal optimization signals, solving the challenges of multi-view geometric inconsistency and data dependency. Specifically, the framework constructs a light-interaction-driven 3DGS reparameterization model to generate multi-scale geometric features consistent with light transport principles, ensuring multi-view normal consistency. A cross-domain feature fusion module is designed within a conditional diffusion model, embedding geometric priors to constrain normal generation while maintaining accurate geometric error propagation. Furthermore, a differentiable 3D reprojection loss strategy is introduced for self-supervised optimization that minimizes geometric error between the reconstructed and input image, eliminating dependence on annotated normal datasets. Quantitative evaluations on the Google Scanned Objects dataset demonstrate that our method outperforms state-of-the-art approaches across multiple metrics.