Controlling Your Image via Simplified Vector Graphics

TL;DR

This paper introduces a novel method for controllable image generation using hierarchical vector graphics, enabling intuitive editing of shapes, colors, and objects with high fidelity.

Contribution

We propose a layer-wise controllable generation framework that parses images into semantic-aligned vector graphics and translates edits into photorealistic outputs.

Findings

Effective image editing and object manipulation demonstrated

High-quality photorealistic outputs from vector graphic edits

Versatile applications in content creation and fine-grained control

Abstract

Recent advances in image generation have achieved remarkable visual quality, while a fundamental challenge remains: Can image generation be controlled at the element level, enabling intuitive modifications such as adjusting shapes, altering colors, or adding and removing objects? In this work, we address this challenge by introducing layer-wise controllable generation through simplified vector graphics (VGs). Our approach first efficiently parses images into hierarchical VG representations that are semantic-aligned and structurally coherent. Building on this representation, we design a novel image synthesis framework guided by VGs, allowing users to freely modify elements and seamlessly translate these edits into photorealistic outputs. By leveraging the structural and semantic features of VGs in conjunction with noise prediction, our method provides precise control over geometry, color, and object semantics. Extensive experiments demonstrate the effectiveness of our approach in diverse applications, including image editing, object-level manipulation, and fine-grained content creation, establishing a new paradigm for controllable image generation. Project page: https://guolanqing.github.io/Vec2Pix/