CAD-Recode: Reverse Engineering CAD Code from Point Clouds

TL;DR

CAD-Recode introduces a method to reverse engineer 3D CAD models from point clouds by translating them into Python code sequences, leveraging LLMs for interpretation and editing.

Contribution

It presents a novel CAD sequence representation as Python code, a network design utilizing LLMs, and a large dataset for training, advancing reverse engineering of CAD models from point clouds.

Findings

Outperforms existing methods on multiple datasets

Generates interpretable Python code for CAD models

Enables CAD editing and question answering from point clouds

Abstract

Computer-Aided Design (CAD) models are typically constructed by sequentially drawing parametric sketches and applying CAD operations to obtain a 3D model. The problem of 3D CAD reverse engineering consists of reconstructing the sketch and CAD operation sequences from 3D representations such as point clouds. In this paper, we address this challenge through novel contributions across three levels: CAD sequence representation, network design, and training dataset. In particular, we represent CAD sketch-extrude sequences as Python code. The proposed CAD-Recode translates a point cloud into Python code that, when executed, reconstructs the CAD model. Taking advantage of the exposure of pre-trained Large Language Models (LLMs) to Python code, we leverage a relatively small LLM as a decoder for CAD-Recode and combine it with a lightweight point cloud projector. CAD-Recode is trained on a procedurally generated dataset of one million CAD sequences. CAD-Recode significantly outperforms existing methods across the DeepCAD, Fusion360 and real-world CC3D datasets. Furthermore, we show that our CAD Python code output is interpretable by off-the-shelf LLMs, enabling CAD editing and CAD-specific question answering from point clouds.