GeoCAD: Local Geometry-Controllable CAD Generation with Large Language Models

TL;DR

GeoCAD introduces a novel method for local geometry-controllable CAD generation using large language models, enabling precise modifications based on user instructions with high quality and consistency.

Contribution

It proposes a new captioning strategy for annotating local CAD parts and leverages LLMs for geometry-aware part prediction, addressing limitations of previous methods.

Findings

High-quality local CAD generation adhering to instructions

Effective annotation of over 221k local parts

Demonstrated consistency between text instructions and generated CAD parts

Abstract

Local geometry-controllable computer-aided design (CAD) generation aims to modify local parts of CAD models automatically, enhancing design efficiency. It also ensures that the shapes of newly generated local parts follow user-specific geometric instructions (e.g., an isosceles right triangle or a rectangle with one corner cut off). However, existing methods encounter challenges in achieving this goal. Specifically, they either lack the ability to follow textual instructions or are unable to focus on the local parts. To address this limitation, we introduce GeoCAD, a user-friendly and local geometry-controllable CAD generation method. Specifically, we first propose a complementary captioning strategy to generate geometric instructions for local parts. This strategy involves vertex-based and VLLM-based captioning for systematically annotating simple and complex parts, respectively. In this way, we caption $\sim$221k different local parts in total. In the training stage, given a CAD model, we randomly mask a local part. Then, using its geometric instruction and the remaining parts as input, we prompt large language models (LLMs) to predict the masked part. During inference, users can specify any local part for modification while adhering to a variety of predefined geometric instructions. Extensive experiments demonstrate the effectiveness of GeoCAD in generation quality, validity and text-to-CAD consistency. Code will be available at https://github.com/Zhanwei-Z/GeoCAD.