Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches

TL;DR

This paper presents Sketch2Model, a view-aware method for reconstructing 3D meshes from single free-hand sketches, addressing sketch ambiguity by explicitly conditioning on viewpoints to enhance reconstruction accuracy.

Contribution

It introduces a novel view-aware approach that explicitly incorporates viewpoint information, improving 3D reconstruction from sketches compared to prior methods.

Findings

Effective in resolving sketch ambiguities

Improves 3D shape reconstruction quality

Works with both predicted and user-specified viewpoints

Abstract

We investigate the problem of generating 3D meshes from single free-hand sketches, aiming at fast 3D modeling for novice users. It can be regarded as a single-view reconstruction problem, but with unique challenges, brought by the variation and conciseness of sketches. Ambiguities in poorly-drawn sketches could make it hard to determine how the sketched object is posed. In this paper, we address the importance of viewpoint specification for overcoming such ambiguities, and propose a novel view-aware generation approach. By explicitly conditioning the generation process on a given viewpoint, our method can generate plausible shapes automatically with predicted viewpoints, or with specified viewpoints to help users better express their intentions. Extensive evaluations on various datasets demonstrate the effectiveness of our view-aware design in solving sketch ambiguities and improving reconstruction quality.