Structural Evaluation Metrics for SVG Generation via Leave-One-Out Analysis

TL;DR

This paper introduces element-level leave-one-out analysis for SVGs, providing structural metrics and quality scores to evaluate and diagnose SVG code beyond visual similarity.

Contribution

It proposes a novel leave-one-out method for SVG analysis, deriving metrics for structural quality and enabling element-level diagnosis and concept attribution.

Findings

Metrics extend SVG evaluation from image similarity to code structure.

Validated on over 19,000 edits across multiple systems and complexity tiers.

Provides zero-shot artifact detection and element-concept attribution.

Abstract

SVG generation is typically evaluated by comparing rendered outputs to reference images, which captures visual similarity but not the structural properties that make SVG editable, decomposable, and reusable. Inspired by the classical jackknife, we introduce element-level leave-one-out (LOO) analysis. The procedure renders the SVG with and without each element, which yields element-level signals for quality assessment and structural analysis. From this single mechanism, we derive (i) per-element quality scores that enable zero-shot artifact detection; (ii) element-concept attribution via LOO footprints crossed with VLM-grounded concept heatmaps; and (iii) four structural metrics: purity, coverage, compactness, and locality, which quantify SVG modularity from complementary angles. These metrics extend SVG evaluation from image similarity to code structure, enabling element-level diagnosis and comparison of how visual concepts are represented, partitioned, and organized within SVG code. Their practical relevance is validated on over 19,000 edits (5 types) across 5 generation systems and 3 complexity tiers.