Reconstruction of a 3D wireframe from a single line drawing via generative depth estimation

TL;DR

This paper introduces a generative depth estimation method using a Latent Diffusion Model to reconstruct 3D wireframes from single line drawings, achieving low average depth error.

Contribution

It presents a novel generative approach with a large dataset and a diffusion model for accurate 3D reconstruction from 2D sketches.

Findings

Achieved 5.3% average depth error in 3D reconstruction.

Demonstrated robustness across various shape complexities.

Trained on over one million image-depth pairs.

Abstract

The conversion of 2D freehand sketches into 3D models remains a pivotal challenge in computer vision, bridging the gap between fluent sketching and CAD. Traditional monocular depth reconstruction techniques are not suitable for line drawing interpretation. We propose a generative approach by framing reconstruction as a conditional dense depth estimation task. To achieve this, we implemented a Latent Diffusion Model (LDM) with a conditioning framework to resolve the inherent ambiguities of orthographic projections. We trained our model using a dataset of over one million image-depth pairs. Our framework demonstrated robust performance across varying shape complexities, with 5.3 percent average depth error.