Neural Surface Priors for Editable Gaussian Splatting

TL;DR

This paper presents a novel method combining neural surface priors with Gaussian Splatting for more flexible and intuitive scene editing from images, enabling seamless updates to appearance through mesh manipulation.

Contribution

It introduces a neural surface prior guiding Gaussian Splatting, allowing for more versatile scene editing by leveraging mesh topology and an improved proxy representation.

Findings

Supports a wider range of scene modifications.

Enables seamless propagation of edits through the proxy.

Improves flexibility over previous methods.

Abstract

In computer graphics and vision, recovering easily modifiable scene appearance from image data is crucial for applications such as content creation. We introduce a novel method that integrates 3D Gaussian Splatting with an implicit surface representation, enabling intuitive editing of recovered scenes through mesh manipulation. Starting with a set of input images and camera poses, our approach reconstructs the scene surface using a neural signed distance field. This neural surface acts as a geometric prior guiding the training of Gaussian Splatting components, ensuring their alignment with the scene geometry. To facilitate editing, we encode the visual and geometric information into a lightweight triangle soup proxy. Edits applied to the mesh extracted from the neural surface propagate seamlessly through this intermediate structure to update the recovered appearance. Unlike previous methods relying on the triangle soup proxy representation, our approach supports a wider range of modifications and fully leverages the mesh topology, enabling a more flexible and intuitive editing process. The complete source code for this project can be accessed at: https://github.com/WJakubowska/NeuralSurfacePriors.