Learning to Synthesize Graphics Programs for Geometric Artworks

TL;DR

This paper introduces Art2Prog, a method that synthesizes executable drawing programs from images, enabling understandable and resolution-independent reproduction of complex artworks by predicting sequences of drawing commands.

Contribution

It presents a novel approach to bridge pixel-level image generation with the actual drawing process through program synthesis, which is a new perspective in art generation research.

Findings

Art2Prog accurately reproduces complex images using drawing commands.

The method demonstrates understanding of higher-level image features.

Generated programs are compact and resolution-independent.

Abstract

Creating and understanding art has long been a hallmark of human ability. When presented with finished digital artwork, professional graphic artists can intuitively deconstruct and replicate it using various drawing tools, such as the line tool, paint bucket, and layer features, including opacity and blending modes. While most recent research in this field has focused on art generation, proposing a range of methods, these often rely on the concept of artwork being represented as a final image. To bridge the gap between pixel-level results and the actual drawing process, we present an approach that treats a set of drawing tools as executable programs. This method predicts a sequence of steps to achieve the final image, allowing for understandable and resolution-independent reproductions under the usage of a set of drawing commands. Our experiments demonstrate that our program synthesizer, Art2Prog, can comprehensively understand complex input images and reproduce them using high-quality executable programs. The experimental results evidence the potential of machines to grasp higher-level information from images and generate compact program-level descriptions.