Content-Aware Texturing for Gaussian Splatting

TL;DR

This paper introduces a content-aware texturing method for Gaussian Splatting that adaptively applies textures to primitives, improving detail representation and reducing parameter count in 3D scene reconstruction.

Contribution

It proposes a novel appearance representation with per-primitive textures that adapt in resolution during optimization, enhancing efficiency and quality in Gaussian Splatting.

Findings

Improved image quality compared to non-textured methods

Reduced number of primitives needed for detailed scenes

Adaptive texture resolution enhances scene detail representation

Abstract

Gaussian Splatting has become the method of choice for 3D reconstruction and real-time rendering of captured real scenes. However, fine appearance details need to be represented as a large number of small Gaussian primitives, which can be wasteful when geometry and appearance exhibit different frequency characteristics. Inspired by the long tradition of texture mapping, we propose to use texture to represent detailed appearance where possible. Our main focus is to incorporate per-primitive texture maps that adapt to the scene in a principled manner during Gaussian Splatting optimization. We do this by proposing a new appearance representation for 2D Gaussian primitives with textures where the size of a texel is bounded by the image sampling frequency and adapted to the content of the input images. We achieve this by adaptively upscaling or downscaling the texture resolution during optimization. In addition, our approach enables control of the number of primitives during optimization based on texture resolution. We show that our approach performs favorably in image quality and total number of parameters used compared to alternative solutions for textured Gaussian primitives. Project page: https://repo-sam.inria.fr/nerphys/gs-texturing/