Deep3DSketch+: Obtaining Customized 3D Model by Single Free-Hand Sketch through Deep Learning

TL;DR

Deep3DSketch+ leverages deep learning to convert single free-hand sketches into detailed 3D models, significantly simplifying and accelerating the design process for manufacturing and customization.

Contribution

Introduces Deep3DSketch+ with view- and structural-aware neural network architecture for high-fidelity 3D model generation from single sketches, overcoming sketch ambiguity and sparsity.

Findings

Achieves state-of-the-art performance on synthetic and real datasets.

Demonstrates successful conversion of sketches into physical 3D objects.

Shows robustness to partial sketch inputs in industrial scenarios.

Abstract

As 3D models become critical in today's manufacturing and product design, conventional 3D modeling approaches based on Computer-Aided Design (CAD) are labor-intensive, time-consuming, and have high demands on the creators. This work aims to introduce an alternative approach to 3D modeling by utilizing free-hand sketches to obtain desired 3D models. We introduce Deep3DSketch+, which is a deep-learning algorithm that takes the input of a single free-hand sketch and produces a complete and high-fidelity model that matches the sketch input. The neural network has view- and structural-awareness enabled by a Shape Discriminator (SD) and a Stroke Enhancement Module (SEM), which overcomes the limitations of sparsity and ambiguity of the sketches. The network design also brings high robustness to partial sketch input in industrial applications.Our approach has undergone extensive experiments, demonstrating its state-of-the-art (SOTA) performance on both synthetic and real-world datasets. These results validate the effectiveness and superiority of our method compared to existing techniques. We have demonstrated the conversion of free-hand sketches into physical 3D objects using additive manufacturing. We believe that our approach has the potential to accelerate product design and democratize customized manufacturing.