Text2Mesh: Text-Driven Neural Stylization for Meshes

TL;DR

Text2Mesh introduces a neural stylization framework that uses text prompts to intuitively control the style of 3D meshes, enabling diverse stylizations without needing pre-trained generative models or UV mappings.

Contribution

The paper presents a novel text-driven neural stylization method for 3D meshes that does not require UV parameterization or pre-trained generative models, and can handle arbitrary mesh qualities.

Findings

Effective stylization of diverse 3D meshes using text prompts.

No need for UV mapping or pre-trained 3D generative models.

Handles low-quality and complex meshes robustly.

Abstract

In this work, we develop intuitive controls for editing the style of 3D objects. Our framework, Text2Mesh, stylizes a 3D mesh by predicting color and local geometric details which conform to a target text prompt. We consider a disentangled representation of a 3D object using a fixed mesh input (content) coupled with a learned neural network, which we term neural style field network. In order to modify style, we obtain a similarity score between a text prompt (describing style) and a stylized mesh by harnessing the representational power of CLIP. Text2Mesh requires neither a pre-trained generative model nor a specialized 3D mesh dataset. It can handle low-quality meshes (non-manifold, boundaries, etc.) with arbitrary genus, and does not require UV parameterization. We demonstrate the ability of our technique to synthesize a myriad of styles over a wide variety of 3D meshes.