Disentangled Geometry and Appearance for Efficient Multi-View Surface Reconstruction and Rendering

TL;DR

This paper introduces a disentangled geometry and appearance model for multi-view surface reconstruction that improves efficiency, quality, and versatility without relying on deep networks, enabling practical editing applications.

Contribution

It proposes a novel explicit mesh-based framework with disentangled geometry and appearance, enhancing learning and broadening application scope compared to prior neural rendering methods.

Findings

Achieves state-of-the-art training and rendering speeds

Provides competitive reconstruction quality

Enables practical mesh and texture editing

Abstract

This paper addresses the limitations of neural rendering-based multi-view surface reconstruction methods, which require an additional mesh extraction step that is inconvenient and would produce poor-quality surfaces with mesh aliasing, restricting downstream applications. Building on the explicit mesh representation and differentiable rasterization framework, this work proposes an efficient solution that preserves the high efficiency of this framework while significantly improving reconstruction quality and versatility. Specifically, we introduce a disentangled geometry and appearance model that does not rely on deep networks, enhancing learning and broadening applicability. A neural deformation field is constructed to incorporate global geometric context, enhancing geometry learning, while a novel regularization constrains geometric features passed to a neural shader to ensure its accuracy and boost shading. For appearance, a view-invariant diffuse term is separated and baked into mesh vertices, further improving rendering efficiency. Experimental results demonstrate that the proposed method achieves state-of-the-art training (4.84 minutes) and rendering (0.023 seconds) speeds, with reconstruction quality that is competitive with top-performing methods. Moreover, the method enables practical applications such as mesh and texture editing, showcasing its versatility and application potential. This combination of efficiency, competitive quality, and broad applicability makes our approach a valuable contribution to multi-view surface reconstruction and rendering.