TreeSketchNet: From Sketch To 3D Tree Parameters Generation

TL;DR

TreeSketchNet is a deep learning system that converts simple, stylized sketches of trees into detailed 3D models by predicting parameters for modeling software, simplifying the modeling process for non-experts.

Contribution

The paper introduces TreeSketchNet, a novel neural network architecture that translates rudimentary tree sketches into 3D model parameters, enabling easier and more accessible 3D tree modeling.

Findings

High accuracy on synthetic sketches

Effective on hand-made sketches

Coherent parameter predictions

Abstract

3D modeling of non-linear objects from stylized sketches is a challenge even for experts in Computer Graphics (CG). The extrapolation of objects parameters from a stylized sketch is a very complex and cumbersome task. In the present study, we propose a broker system that mediates between the modeler and the 3D modelling software and can transform a stylized sketch of a tree into a complete 3D model. The input sketches do not need to be accurate or detailed, and only need to represent a rudimentary outline of the tree that the modeler wishes to 3D-model. Our approach is based on a well-defined Deep Neural Network (DNN) architecture, we called TreeSketchNet (TSN), based on convolutions and able to generate Weber and Penn parameters that can be interpreted by the modelling software to generate a 3D model of a tree starting from a simple sketch. The training dataset consists of Synthetically-Generated \revision{(SG)} sketches that are associated with Weber-Penn parameters generated by a dedicated Blender modelling software add-on. The accuracy of the proposed method is demonstrated by testing the TSN with both synthetic and hand-made sketches. Finally, we provide a qualitative analysis of our results, by evaluating the coherence of the predicted parameters with several distinguishing features.