Vector Scaffolding: Inter-Scale Orchestration for Differentiable Image Vectorization

TL;DR

This paper introduces Vector Scaffolding, a hierarchical optimization framework for differentiable image vectorization that improves stability, speed, and quality over flat pixel-matching methods.

Contribution

It proposes a novel topological construction approach with interior gradient aggregation and progressive densification to enhance vectorization.

Findings

Accelerates optimization by 2.5 times.

Improves PSNR by up to 1.4 dB.

Reduces topology collapse and uneditable polygon soup.

Abstract

Differentiable vector graphics have enabled powerful gradient-based optimization of vector primitives directly from raster images. However, existing frameworks formulate this as a flat optimization problem, forcing hundreds to thousands of randomly initialized curves to blindly compete for pixel-level error reduction. This disordered optimization leads to topology collapse, where macroscopic structures are distorted by internal high-frequency noise, resulting in a redundant and uneditable "polygon soup" that limits practical editability. To address this limitation, we propose Vector Scaffolding, a novel hierarchical optimization framework that shifts from flat pixel-matching to structured topological construction tailored for vector graphics. By identifying a key cause of topology collapse as the mathematical imbalance between area and boundary gradients, we introduce Interior Gradient Aggregation to stabilize the learning dynamics of multi-scale curve mixtures. Upon this stabilized landscape, we employ Progressive Stratification and Rapid Inflation Scheduling to progressively densify vector primitives with extremely high learning rates ($\times 50$). Experiments demonstrate that our approach accelerates optimization by $2.5\times$ while simultaneously improving PSNR by up to 1.4 dB over the previous state of the art.