Automatic Reverse Engineering: Creating computer-aided design (CAD) models from multi-view images

TL;DR

This paper introduces a novel neural network architecture for automated reverse engineering of CAD models from multi-view images, reducing manual effort and improving accuracy over traditional point-cloud methods.

Contribution

The paper presents a new multi-stage neural network combining CNNs, multi-view pooling, and transformers for automated CAD model generation from images.

Findings

Successfully reconstructs CAD models from simulated images.

Outperforms a state-of-the-art point-based network in accuracy metrics.

Demonstrates transferability to real-world photographs with limitations.

Abstract

Generation of computer-aided design (CAD) models from multi-view images may be useful in many practical applications. To date, this problem is usually solved with an intermediate point-cloud reconstruction and involves manual work to create the final CAD models. In this contribution, we present a novel network for an automated reverse engineering task. Our network architecture combines three distinct stages: A convolutional neural network as the encoder stage, a multi-view pooling stage and a transformer-based CAD sequence generator. The model is trained and evaluated on a large number of simulated input images and extensive optimization of model architectures and hyper-parameters is performed. A proof-of-concept is demonstrated by successfully reconstructing a number of valid CAD models from simulated test image data. Various accuracy metrics are calculated and compared to a state-of-the-art point-based network. Finally, a real world test is conducted supplying the network with actual photographs of two three-dimensional test objects. It is shown that some of the capabilities of our network can be transferred to this domain, even though the training exclusively incorporates purely synthetic training data. However to date, the feasible model complexity is still limited to basic shapes.