TeSO: Representing and Compressing 3D Point Cloud Scenes with Textured Surfel Octree

TL;DR

TeSO introduces a novel 3D scene representation using textured surfel octrees that enhances rendering quality and compression efficiency for 3D content streaming applications.

Contribution

The paper proposes Textured Surfel Octree (TeSO), a new 3D representation combining surfels with textures organized in an octree, improving quality and compressibility over existing methods.

Findings

Achieves higher rendering quality at lower bit-rates.

Effectively captures high-frequency texture details.

Outperforms baseline methods in compression efficiency.

Abstract

3D visual content streaming is a key technology for emerging 3D telepresence and AR/VR applications. One fundamental element underlying the technology is a versatile 3D representation that is capable of producing high-quality renders and can be efficiently compressed at the same time. Existing 3D representations like point clouds, meshes and 3D Gaussians each have limitations in terms of rendering quality, surface definition, and compressibility. In this paper, we present the Textured Surfel Octree (TeSO), a novel 3D representation that is built from point clouds but addresses the aforementioned limitations. It represents a 3D scene as cube-bounded surfels organized on an octree, where each surfel is further associated with a texture patch. By approximating a smooth surface with a large surfel at a coarser level of the octree, it reduces the number of primitives required to represent the 3D scene, and yet retains the high-frequency texture details through the texture map attached to each surfel. We further propose a compression scheme to encode the geometry and texture efficiently, leveraging the octree structure. The proposed textured surfel octree combined with the compression scheme achieves higher rendering quality at lower bit-rates compared to multiple point cloud and 3D Gaussian-based baselines.