ShapeLib: Designing a library of programmatic 3D shape abstractions with Large Language Models

TL;DR

ShapeLib leverages large language models to automatically design and validate reusable 3D shape abstraction libraries based on text descriptions and exemplar shapes, improving shape analysis and editing.

Contribution

This work introduces ShapeLib, a novel framework that combines LLMs with geometric reasoning to discover and validate 3D shape abstractions from minimal input.

Findings

LLMs can generate generalizable shape abstractions

ShapeLib outperforms prior methods in usability and plausibility

Abstractions enable advanced shape editing and generation

Abstract

We present ShapeLib, the first method that leverages the priors of LLMs to design libraries of programmatic 3D shape abstractions. Our system accepts two forms of design intent: text descriptions of functions to include in the library and a seed set of exemplar shapes. We discover abstractions that match this design intent with a guided LLM workflow that first proposes, and then validates, different ways of applying and implementing functions. We learn recognition networks that map shapes to programs with these newly discovered abstractions by training on data produced by LLM authored synthetic data generation procedures. Across modeling domains (split by shape category), we find that LLMs, when thoughtfully combined with geometric reasoning, can be guided to author a library of abstraction functions that generalize to shapes outside of the seed set. This framework addresses a long-standing shape analysis problem of how to discover reusable abstraction functions while exposing interpretable, semantically aligned interfaces. We find that ShapeLib provides distinct advantages over prior alternative abstraction discovery works in terms of generalization, usability, and maintaining plausibility under manipulation. Finally, we demonstrate that ShapeLib's abstraction functions unlock a number of downstream applications, combining LLM reasoning over shape programs with geometry processing to support shape editing and generation.