Radiant Triangle Soup with Soft Connectivity Forces for 3D Reconstruction and Novel View Synthesis

TL;DR

This paper presents a scene optimization method using a collection of translucent triangles with soft connectivity forces, improving 3D reconstruction and view synthesis accuracy while maintaining visual quality.

Contribution

It introduces a novel triangle soup representation with soft connectivity forces for enhanced 3D scene modeling and synthesis.

Findings

Improved geometric accuracy over state-of-the-art methods.

Effective incorporation of soft connectivity for surface continuity.

Maintains high visual fidelity in reconstructions.

Abstract

We introduce an inference-time scene optimization algorithm utilizing triangle soup, a collection of disconnected translucent triangle primitives, as the representation for the geometry and appearance of a scene. Unlike full-rank Gaussian kernels, triangles are a natural, locally-flat proxy for surfaces that can be connected to achieve highly complex geometry. When coupled with per-vertex Spherical Harmonics (SH), triangles provide a rich visual representation without incurring an expensive increase in primitives. We leverage our new representation to incorporate optimization objectives and enforce spatial regularization directly on the underlying primitives. The main differentiator of our approach is the definition and enforcement of soft connectivity forces between triangles during optimization, encouraging explicit, but soft, surface continuity in 3D. Experiments on representative 3D reconstruction and novel view synthesis datasets show improvements in geometric accuracy compared to current state-of-the-art algorithms without sacrificing visual fidelity.