neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing

TL;DR

neuralCAD-Edit is a new benchmark for 3D CAD model editing based on expert interactions, revealing significant performance gaps between foundation models and human CAD engineers.

Contribution

introduces the first expert-annotated benchmark for multimodal 3D CAD editing, highlighting challenges for current foundation models.

Findings

foundation models lag 53% behind human experts in acceptance trials

neuralCAD-Edit captures realistic expert editing requests

benchmark reveals substantial performance gap in automatic and human evaluations

Abstract

We introduce neuralCAD-Edit, the first benchmark for editing 3D CAD models collected from expert CAD engineers. Instead of text conditioning as in prior works, we collect realistic CAD editing requests by capturing videos of professional designers, interacting directly with CAD models in CAD software, while talking, pointing and drawing. We recruited ten consenting designers to contribute to this contained study. We benchmark leading foundation models against human CAD experts carrying out edits, and find a large performance gap in both automatic metrics and human evaluations. Even the best foundation model (GPT 5.2) scores 53% lower (absolute) than CAD experts in human acceptance trials, demonstrating the challenge of neuralCAD-Edit. We hope neuralCAD-Edit will provide a solid foundation against which 3D CAD editing approaches and foundation models can be developed. Code/data: https://autodeskailab.github.io/neuralCAD-Edit