Semantic Compression of 3D Objects for Open and Collaborative Virtual Worlds

TL;DR

This paper introduces a semantic compression method for 3D objects using natural language and deep generative models, achieving higher compression rates than traditional structural methods while maintaining quality.

Contribution

It presents a novel pipeline for 3D semantic compression leveraging public generative models and explores its effectiveness at extreme compression rates.

Findings

Achieved up to 105x compression rates on 3D objects.

Semantic compression outperforms traditional methods around 100x compression.

Demonstrated the feasibility of human-readable, language-based 3D object storage.

Abstract

Traditional methods for 3D object compression operate only on structural information within the object vertices, polygons, and textures. These methods are effective at compression rates up to 10x for standard object sizes but quickly deteriorate at higher compression rates with texture artifacts, low-polygon counts, and mesh gaps. In contrast, semantic compression ignores structural information and operates directly on the core concepts to push to extreme levels of compression. In addition, it uses natural language as its storage format, which makes it natively human-readable and a natural fit for emerging applications built around large-scale, collaborative projects within augmented and virtual reality. It deprioritizes structural information like location, size, and orientation and predicts the missing information with state-of-the-art deep generative models. In this work, we construct a pipeline for 3D semantic compression from public generative models and explore the quality-compression frontier for 3D object compression. We apply this pipeline to achieve rates as high as 105x for 3D objects taken from the Objaverse dataset and show that semantic compression can outperform traditional methods in the important quality-preserving region around 100x compression.