Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches

TL;DR

This paper introduces a novel encoder/decoder framework for reconstructing and editing 3D shapes from sparse sketches, enabling effective shape refinement and outperforming existing methods.

Contribution

It presents a new sketch-to-mesh translation method that leverages latent space for shape refinement, robust to style variations and capable of single-stroke editing.

Findings

Outperforms state-of-the-art sketch-based 3D reconstruction methods

Robust to different sketch styles and single-stroke inputs

Effective for shape refinement and editing

Abstract

Reconstructing 3D shape from 2D sketches has long been an open problem because the sketches only provide very sparse and ambiguous information. In this paper, we use an encoder/decoder architecture for the sketch to mesh translation. When integrated into a user interface that provides camera parameters for the sketches, this enables us to leverage its latent parametrization to represent and refine a 3D mesh so that its projections match the external contours outlined in the sketch. We will show that this approach is easy to deploy, robust to style changes, and effective. Furthermore, it can be used for shape refinement given only single pen strokes. We compare our approach to state-of-the-art methods on sketches -- both hand-drawn and synthesized -- and demonstrate that we outperform them.