Reflective Gaussian Splatting
Yuxuan Yao, Zixuan Zeng, Chun Gu, Xiatian Zhu, Li Zhang
TL;DR
Ref-Gaussian introduces a novel real-time rendering framework for reflective objects that combines physically based deferred rendering and Gaussian-grounded inter-reflection, achieving high-quality results and supporting various applications.
Contribution
It is the first to integrate Gaussian splatting with inter-reflection modeling and physically based rendering for reflective scene synthesis.
Findings
Outperforms existing methods in quality and efficiency
Supports both reflective and non-reflective scene rendering
Enables applications like relighting and editing
Abstract
Novel view synthesis has experienced significant advancements owing to increasingly capable NeRF- and 3DGS-based methods. However, reflective object reconstruction remains challenging, lacking a proper solution to achieve real-time, high-quality rendering while accommodating inter-reflection. To fill this gap, we introduce a Reflective Gaussian splatting (Ref-Gaussian) framework characterized with two components: (I) Physically based deferred rendering that empowers the rendering equation with pixel-level material properties via formulating split-sum approximation; (II) Gaussian-grounded inter-reflection that realizes the desired inter-reflection function within a Gaussian splatting paradigm for the first time. To enhance geometry modeling, we further introduce material-aware normal propagation and an initial per-Gaussian shading stage, along with 2D Gaussian primitives. Extensive…
Peer Reviews
Decision·ICLR 2025 Poster
1. By extracting explicit geometry, the paper addresses the inter-reflection issue in Gaussian splatting, which is important in realistic PBR. Experimental results showcase its SOTA performance in novel view synthesis and decomposition on reflective cases. 2. Instead of per-Gaussian PBR (like Relightable 3DGS or Gaussian Shader), The proposed method employs an effective PBR deferred rendering to achieve better PBR performance (similar to 3DGS-DR). Further ablation study demonstrates the superior
1. The effectiveness of inter-reflection technique lacks further qualitative evidence (e.g. providing indirect components in Fig.5 or showcasing indirect components in Ref-Real dataset where the multiple objects provide rich inter-reflection), as the ablation study in table 3 indicates only a slight decrease in PSNR when rendering without inter-reflection. 2. The ablation study in table 3 only takes PSNR changes into account, while the influences on geometry may need further demonstration (e.g.
- Rendering quality is excellent, and reconstructed normals are accurate. - Training time and rendering speed are satisfactory
- The contribution of this paper may lack novelty. For the deferred shading part of the pipeline, I think many previous 3DGS-based inverse rendering methods[1][2][3] have adopted these techniques and [2][3] also use split-sum approximation to handle the intractable rendering equation. Further more, assigning each Gaussian with a new attribute to model the indirect lighting has also been proposed in previous methods[4]. The innovation of this method lies in the new visibility modeling scheme and
1. The key designs proposed by the paper are rational. It is not surprising that leveraging physically-based rendering can significanty improve the performance of specular rendering. 2. I like the idea of material-aware normal propagation. Seems like it can greatly improve the quality of surface normal reconstruction. 3. Extensive qualitative and quantitative experiments demonstrate that the proposed method outperforms baselines. 4. I appreciate the comprehensive ablation study that covers lots
1. Missing details. L233-234: "During optimization, we periodically extract the object’s surface mesh using truncated signed distance function (TSDF) fusion." The authors need to specify the number of steps as the period of mesh extraction. 2. Unclear explanations. I have several confusions when reading the paper, and please see the "Questions" part. I believe that the paper writing can be improved to explain the method more clearly. 3. Baselines. Although this method is based on 3DGS, it should
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemiconductor Lasers and Optical Devices