3DSS: 3D Surface Splatting for Inverse Rendering

TL;DR

3DSS introduces a differentiable surface splatting renderer enabling physically-based inverse rendering from multi-view images, jointly recovering shape, materials, and lighting with high-quality results.

Contribution

It is the first to formulate a differentiable surface splatting renderer that directly models reconstruction kernels for inverse rendering tasks.

Findings

Achieves anti-aliased silhouettes and visibility gradients at sparse edges.

Jointly recovers shape, BRDF materials, and illumination.

Outperforms mesh-based, implicit, and Gaussian-splatting baselines.

Abstract

We present 3D Surface Splatting (3DSS), the first differentiable surface splatting renderer for physically-based inverse rendering from multi-view images. Our central insight is that the surface separation problem at the heart of surface splatting admits a direct formulation in terms of the reconstruction kernels themselves. From this foundation we derive a coverage-based compositing model whose per-layer opacity arises directly from the accumulated Elliptical Weighted Average reconstruction weight, yielding anti-aliased silhouettes and informative visibility gradients at sparsely covered edges. Combined with forward microfacet shading under co-optimized HDR environment lighting and density-aware adaptive refinement, 3DSS jointly recovers shape, spatially-varying BRDF materials, and illumination. Because the optimized representation is a set of oriented surface samples, it bridges natively to mesh-based workflows via surface reconstruction from oriented point cloud methods. We evaluate 3DSS against mesh-based, implicit, and Gaussian-splatting baselines across geometry reconstruction, novel-view synthesis, and novel-illumination relighting.