Learning to Build Shapes by Extrusion

TL;DR

This paper presents a novel text-based mesh representation called TEE, enabling mesh generation, editing, and synthesis using large language models, supporting arbitrary face counts and manifold meshes.

Contribution

Introducing TEE, a text-encoded extrusion representation for mesh construction, and a method for generating and editing 3D meshes with LLMs, improving flexibility and manifold guarantees.

Findings

The method supports arbitrary face counts in generated meshes.

Meshes can be reconstructed, synthesized, and edited using the learned extrusion sequences.

The approach produces manifold meshes by design, unlike previous transformer models.

Abstract

We introduce Text Encoded Extrusions (TEE), a text-based representation that expresses mesh construction as sequences of face extrusions rather than polygon lists, and a method for generating 3D meshes from TEE using a large language model (LLM). By learning extrusion sequences that assemble a mesh, similar to the way artists create meshes, our approach naturally supports arbitrary output face counts and produces manifold meshes by design, in contrast to recent mesh generative transformer based models. The learnt extrusion sequences can also be applied to existing meshes - enabling editing in addition to generation. To train our model, we decompose a library of quadrilateral meshes with non-self-intersecting face loops into constituent loops, which can be viewed as their building blocks, and finetune an LLM on the steps for reassembling the quadrilateral meshes by performing a sequence of extrusions. We demonstrate that our representation enables reconstruction, novel shape synthesis, and the addition of new features to existing meshes.