Sketch2MinSurf: Vision-Language Guided Generation of Editable Minimal Surfaces from Hand-Drawn Sketches

TL;DR

Sketch2MinSurf introduces a hybrid vision-language and geometric optimization method to convert hand-drawn sketches into smooth, editable 3D minimal surfaces, ensuring topological consistency and usability in design workflows.

Contribution

It presents a novel spatial-topological encoding and a structural loss function to generate topologically coherent, editable minimal surfaces from sketches, outperforming existing methods.

Findings

Achieves a topological similarity score of 0.844 on a test set of 100 sketches.

Generates manifolds that are directly editable and free from non-manifold artifacts.

Demonstrates potential for human-driven 3D form generation in art installations.

Abstract

Converting hand-drawn sketches into structured 3D geometries remains challenging due to the difficulty of representing non-Euclidean surfaces and maintaining topological consistency. Existing generative models such as GANs, NeRFs, and diffusion architectures often fail to produce editable manifolds directly usable in downstream design workflows. We present Sketch2MinSurf, a hybrid vision-language and geometric optimization framework that integrates vision-language guidance with minimal-surface theory to generate smooth and editable 3D surfaces from hand-drawn sketches. The core of our approach is a spatial-topological encoding that represents geometry as tuples of node coordinates and real/virtual edge skeletons, enabling stable topological control during generation. We further introduce the Sketch2MinSurf Structural Loss (S2MS-Loss), a reward-modulated objective that jointly constrains geometric reconstruction and topological coherence. On a test set of 100 sketches, Sketch2MinSurf achieves a topological similarity score of 0.844, outperforming existing sketch-to-shape baselines. The generated manifolds are directly editable and free from non-manifold artifacts. A public art installation at a university showcases the method's potential for human-intent-driven 3D form generation. The dataset and code are available at https://anonymous.4open.science/r/Sketch2MinSurf/.