Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes

TL;DR

This paper presents a novel framework that converts abstract sketches into precise 3D shapes by leveraging part-level modelling, cross-modal correspondence, and low-dimensional implicit spaces, making 3D content creation more accessible and efficient.

Contribution

It introduces a part-level modelling and alignment framework that enables sketch-to-3D shape conversion without paired datasets, enhancing accessibility and efficiency in 3D content creation.

Findings

Effective conversion of sketches to 3D shapes demonstrated.

Reduced computational demands through low-dimensional implicit space.

Seamless in-position editing enabled by cross-modal part alignment.

Abstract

In this paper, we democratise 3D content creation, enabling precise generation of 3D shapes from abstract sketches while overcoming limitations tied to drawing skills. We introduce a novel part-level modelling and alignment framework that facilitates abstraction modelling and cross-modal correspondence. Leveraging the same part-level decoder, our approach seamlessly extends to sketch modelling by establishing correspondence between CLIPasso edgemaps and projected 3D part regions, eliminating the need for a dataset pairing human sketches and 3D shapes. Additionally, our method introduces a seamless in-position editing process as a byproduct of cross-modal part-aligned modelling. Operating in a low-dimensional implicit space, our approach significantly reduces computational demands and processing time.