Enhancement of 3D Gaussian Splatting using Raw Mesh for Photorealistic Recreation of Architectures

TL;DR

This paper introduces a method that leverages raw 3D models to enhance 3D Gaussian Splatting, resulting in more accurate and visually detailed architectural reconstructions from images.

Contribution

It proposes a novel approach to incorporate raw 3D models into Gaussian Splatting, improving shape accuracy and visual quality in architectural scene reconstruction.

Findings

Enhanced shape capture of buildings using raw 3D models.

Improved visual quality of textures and details.

Potential applications in architectural design and preservation.

Abstract

The photorealistic reconstruction and rendering of architectural scenes have extensive applications in industries such as film, games, and transportation. It also plays an important role in urban planning, architectural design, and the city's promotion, especially in protecting historical and cultural relics. The 3D Gaussian Splatting, due to better performance over NeRF, has become a mainstream technology in 3D reconstruction. Its only input is a set of images but it relies heavily on geometric parameters computed by the SfM process. At the same time, there is an existing abundance of raw 3D models, that could inform the structural perception of certain buildings but cannot be applied. In this paper, we propose a straightforward method to harness these raw 3D models to guide 3D Gaussians in capturing the basic shape of the building and improve the visual quality of textures and details when photos are captured non-systematically. This exploration opens up new possibilities for improving the effectiveness of 3D reconstruction techniques in the field of architectural design.