Hierarchical SVG Tokenization: Learning Compact Visual Programs for Scalable Vector Graphics Modeling

TL;DR

HiVG introduces a hierarchical SVG tokenization framework that enhances vector graphics generation by improving sequence efficiency and spatial consistency through structured tokens and a novel initialization strategy.

Contribution

The paper presents HiVG, a hierarchical tokenization method and training paradigm that significantly improves SVG program synthesis over traditional byte-level approaches.

Findings

Enhanced generation fidelity and spatial consistency in SVG outputs.

Improved sequence efficiency and reduced token redundancy.

Better stability in learning executable SVG programs.

Abstract

Recent large language models have shifted SVG generation from differentiable rendering optimization to autoregressive program synthesis. However, existing approaches still rely on generic byte-level tokenization inherited from natural language processing, which poorly reflects the geometric structure of vector graphics. Numerical coordinates are fragmented into discrete symbols, destroying spatial relationships and introducing severe token redundancy, often leading to coordinate hallucination and inefficient long-sequence generation. To address these challenges, we propose HiVG, a hierarchical SVG tokenization framework tailored for autoregressive vector graphics generation. HiVG decomposes raw SVG strings into structured \textit{atomic tokens} and further compresses executable command--parameter groups into geometry-constrained \textit{segment tokens}, substantially improving sequence efficiency while preserving syntactic validity. To further mitigate spatial mismatch, we introduce a Hierarchical Mean--Noise (HMN) initialization strategy that injects numerical ordering signals and semantic priors into new token embeddings. Combined with a curriculum training paradigm that progressively increases program complexity, HiVG enables more stable learning of executable SVG programs. Extensive experiments on both text-to-SVG and image-to-SVG tasks demonstrate improved generation fidelity, spatial consistency, and sequence efficiency compared with conventional tokenization schemes. Our code is publicly available at https://github.com/ximinng/HiVG