Text-to-Vector Generation with Neural Path Representation

TL;DR

This paper introduces a novel neural path representation and a two-stage optimization process for text-to-vector generation, improving geometric constraints and visual quality of SVGs using a dual-branch VAE and diffusion guidance.

Contribution

It proposes a neural path representation with a dual-branch VAE and a two-stage optimization, enhancing geometric constraints and visual quality in text-to-vector generation.

Findings

Effective incorporation of geometric constraints in SVG generation.

Improved visual and topological quality of generated vector graphics.

Versatile applications demonstrated through extensive experiments.

Abstract

Vector graphics are widely used in digital art and highly favored by designers due to their scalability and layer-wise properties. However, the process of creating and editing vector graphics requires creativity and design expertise, making it a time-consuming task. Recent advancements in text-to-vector (T2V) generation have aimed to make this process more accessible. However, existing T2V methods directly optimize control points of vector graphics paths, often resulting in intersecting or jagged paths due to the lack of geometry constraints. To overcome these limitations, we propose a novel neural path representation by designing a dual-branch Variational Autoencoder (VAE) that learns the path latent space from both sequence and image modalities. By optimizing the combination of neural paths, we can incorporate geometric constraints while preserving expressivity in generated SVGs. Furthermore, we introduce a two-stage path optimization method to improve the visual and topological quality of generated SVGs. In the first stage, a pre-trained text-to-image diffusion model guides the initial generation of complex vector graphics through the Variational Score Distillation (VSD) process. In the second stage, we refine the graphics using a layer-wise image vectorization strategy to achieve clearer elements and structure. We demonstrate the effectiveness of our method through extensive experiments and showcase various applications. The project page is https://intchous.github.io/T2V-NPR.