StructuredField: Unifying Structured Geometry and Radiance Field

TL;DR

StructuredField introduces a novel structured tetrahedral mesh representation for high-fidelity, differentiable rendering that integrates well with existing graphics pipelines and simulation algorithms, ensuring geometric integrity and multi-scale detail capture.

Contribution

The paper presents a new structured mesh-based representation with a conformal, inversion-free tetrahedral mesh and a reparameterization into 3D Gaussian primitives for differentiable rendering.

Findings

Achieves high-quality, inversion-free, conformal tetrahedral meshes.

Provides comparable reconstruction quality to existing methods like 3DGS.

Demonstrates applicability to physical simulation, deformation, and level-of-detail tasks.

Abstract

Recent point-based differentiable rendering techniques have achieved significant success in high-fidelity reconstruction and fast rendering. However, due to the unstructured nature of point-based representations, they are difficult to apply to modern graphics pipelines designed for structured meshes, as well as to a variety of simulation and editing algorithms that work well with structured mesh representations. To this end, we propose StructuredField, a novel representation that achieves both a structured geometric representation of the reconstructed object and high-fidelity rendering reconstruction. We employ structured tetrahedral meshes to represent the reconstructed object. We reparameterize the geometric attributes of these tetrahedra into the parameters of 3D Gaussian primitives, thereby enabling differentiable, high-fidelity rendering directly from the mesh. Furthermore, a hierarchical implicit subdivision strategy is utilized to ensure a conformal mesh structure while empowering the representation to capture multi-scale details. To maintain geometric integrity during optimization, we propose a novel inversion-free homeomorphism that constrains the tetrahedral mesh, guaranteeing it remains both inversion-free and self-intersection-free during the optimization process and in the final result. Based on our proposed StructuredField, we achieve high-quality structured meshes that are completely inversion-free and conformal, while also attaining reconstruction results comparable to those of 3DGS. We also demonstrate the applicability of our representation to various applications such as physical simulation, deformation, and level-of-detail.