Mesh Splatting for End-to-end Multiview Surface Reconstruction

TL;DR

This paper introduces a differentiable mesh-to-volume conversion method that enables end-to-end multiview surface reconstruction, improving mesh quality and capturing complex geometries efficiently.

Contribution

It proposes a novel approach to soften meshes into semi-transparent layers for volumetric rendering, bridging the gap between volumetric and surface methods.

Findings

Achieves accurate surface reconstruction in about 20 minutes.

Substantially improves mesh quality over traditional methods.

Effectively models complex geometries with controllable receptive fields.

Abstract

Surfaces are typically represented as meshes, which can be extracted from volumetric fields via meshing or optimized directly as surface parameterizations. Volumetric representations occupy 3D space and have a large effective receptive field along rays, enabling stable and efficient optimization via volumetric rendering; however, subsequent meshing often produces overly dense meshes and introduces accumulated errors. In contrast, pure surface methods avoid meshing but capture only boundary geometry with a single-layer receptive field, making it difficult to learn intricate geometric details and increasing reliance on priors (e.g., shading or normals). We bridge this gap by differentiably turning a surface representation into a volumetric one, enabling end-to-end surface reconstruction via volumetric rendering to model complex geometries. Specifically, we soften a mesh into multiple semi-transparent layers that remain differentiable with respect to the base mesh, endowing it with a controllable 3D receptive field. Combined with a splatting-based renderer and a topology-control strategy, our method can be optimized in about 20 minutes to achieve accurate surface reconstruction while substantially improving mesh quality.