Deep Learning Assisted Optimization for 3D Reconstruction from Single 2D Line Drawings

TL;DR

This paper introduces a deep learning approach to enhance the accuracy and success rate of 3D reconstruction from single line drawings by predicting geometric relationships and initial depths.

Contribution

It presents a novel method combining deep neural networks with traditional optimization techniques for improved 3D reconstruction from 2D line drawings.

Findings

Significantly improved success rate in 3D reconstruction.

Effective detection of geometric relationships among edges.

Enhanced initial depth predictions lead to better optimization outcomes.

Abstract

In this paper, we revisit the long-standing problem of automatic reconstruction of 3D objects from single line drawings. Previous optimization-based methods can generate compact and accurate 3D models, but their success rates depend heavily on the ability to (i) identifying a sufficient set of true geometric constraints, and (ii) choosing a good initial value for the numerical optimization. In view of these challenges, we propose to train deep neural networks to detect pairwise relationships among geometric entities (i.e., edges) in the 3D object, and to predict initial depth value of the vertices. Our experiments on a large dataset of CAD models show that, by leveraging deep learning in a geometric constraint solving pipeline, the success rate of optimization-based 3D reconstruction can be significantly improved.