LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion

TL;DR

This paper presents LaGeM, a hierarchical autoencoder for 3D models that efficiently compresses and represents complex geometries, enabling improved generative modeling with diffusion techniques on unordered vector sets.

Contribution

Introduces a hierarchical autoencoder for 3D models operating on unordered vectors, enabling efficient large-scale representation and diffusion-based generative modeling.

Findings

Model achieves high-resolution geometry representation.

Training requires less time and memory than baseline.

Effective for generative modeling with cascaded diffusion.

Abstract

This paper introduces a novel hierarchical autoencoder that maps 3D models into a highly compressed latent space. The hierarchical autoencoder is specifically designed to tackle the challenges arising from large-scale datasets and generative modeling using diffusion. Different from previous approaches that only work on a regular image or volume grid, our hierarchical autoencoder operates on unordered sets of vectors. Each level of the autoencoder controls different geometric levels of detail. We show that the model can be used to represent a wide range of 3D models while faithfully representing high-resolution geometry details. The training of the new architecture takes 0.70x time and 0.58x memory compared to the baseline. We also explore how the new representation can be used for generative modeling. Specifically, we propose a cascaded diffusion framework where each stage is conditioned on the previous stage. Our design extends existing cascaded designs for image and volume grids to vector sets.