Creating Language-driven Spatial Variations of Icon Images

TL;DR

This paper introduces a language-driven method for editing icon images to produce spatial variations, enabling automatic, coherent modifications based on user prompts, which outperforms existing approaches.

Contribution

The method translates user requests into geometric constraints using a large language model and optimizes segment transformations to generate spatially varied icons.

Findings

Outperforms multiple baselines in producing spatial variations

Enables natural and coherent icon editing from text prompts

Both quantitative and qualitative results validate effectiveness

Abstract

Editing 2D icon images can require significant manual effort from designers. It involves manipulating multiple geometries while maintaining the logical or physical coherence of the objects depicted in the image. Previous language driven image editing methods can change the texture and geometry of objects in the image but fail at producing spatial variations, i.e. modifying spatial relations between objects while maintaining their identities. We present a language driven editing method that can produce spatial variations of icon images. Our method takes in an icon image along with a user's editing request text prompt and outputs an edited icon image reflecting the user's editing request. Our method is designed based on two key observations: (1) A user's editing requests can be translated by a large language model (LLM), with help from a domain specific language (DSL) library, into to a set of geometrical constraints defining the relationships between segments in an icon image. (2) Optimizing the affine transformations of the segments with respect to these geometrical constraints can produce icon images that fulfill the editing request and preserve overall physical and logical coherence. Quantitative and qualitative results show that our system outperforms multiple baselines, enabling natural editing of icon images.