3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations

TL;DR

3DStyleNet is a novel method that creates stylized 3D shapes by transferring geometric and texture styles from source to target objects, enabling easy 3D content creation and data augmentation.

Contribution

The paper introduces a two-stage neural network approach for stylizing 3D shapes with geometric and texture variations, combining shape warping and style transfer techniques.

Findings

Outperforms existing data augmentation methods in 3D reconstruction tasks.

Generates diverse stylized 3D shapes from limited high-quality examples.

Validated through qualitative results and user studies.

Abstract

We propose a method to create plausible geometric and texture style variations of 3D objects in the quest to democratize 3D content creation. Given a pair of textured source and target objects, our method predicts a part-aware affine transformation field that naturally warps the source shape to imitate the overall geometric style of the target. In addition, the texture style of the target is transferred to the warped source object with the help of a multi-view differentiable renderer. Our model, 3DStyleNet, is composed of two sub-networks trained in two stages. First, the geometric style network is trained on a large set of untextured 3D shapes. Second, we jointly optimize our geometric style network and a pre-trained image style transfer network with losses defined over both the geometry and the rendering of the result. Given a small set of high-quality textured objects, our method can create many novel stylized shapes, resulting in effortless 3D content creation and style-ware data augmentation. We showcase our approach qualitatively on 3D content stylization, and provide user studies to validate the quality of our results. In addition, our method can serve as a valuable tool to create 3D data augmentations for computer vision tasks. Extensive quantitative analysis shows that 3DStyleNet outperforms alternative data augmentation techniques for the downstream task of single-image 3D reconstruction.