A Style is Worth One Code: Unlocking Code-to-Style Image Generation with Discrete Style Space

TL;DR

This paper introduces CoTyle, an open-source method that generates consistent and novel visual styles from a numerical code, enabling flexible style control in image generation with a simple code-to-style approach.

Contribution

The paper presents the first open-source framework for code-to-style image generation, utilizing a discrete style codebook and a style generator to produce diverse, consistent styles from minimal input.

Findings

CoTyle effectively maps numerical codes to visual styles.

The method produces diverse and consistent stylized images.

It demonstrates the potential for simple style control in generative models.

Abstract

Innovative visual stylization is a cornerstone of artistic creation, yet generating novel and consistent visual styles remains a significant challenge. Existing generative approaches typically rely on lengthy textual prompts, reference images, or parameter-efficient fine-tuning to guide style-aware image generation, but often struggle with style consistency, limited creativity, and complex style representations. In this paper, we affirm that a style is worth one numerical code by introducing the novel task, code-to-style image generation, which produces images with novel, consistent visual styles conditioned solely on a numerical style code. To date, this field has only been primarily explored by the industry (e.g., Midjourney), with no open-source research from the academic community. To fill this gap, we propose CoTyle, the first open-source method for this task. Specifically, we first train a discrete style codebook from a collection of images to extract style embeddings. These embeddings serve as conditions for a text-to-image diffusion model (T2I-DM) to generate stylistic images. Subsequently, we train an autoregressive style generator on the discrete style embeddings to model their distribution, allowing the synthesis of novel style embeddings. During inference, a numerical style code is mapped to a unique style embedding by the style generator, and this embedding guides the T2I-DM to generate images in the corresponding style. Unlike existing methods, our method offers unparalleled simplicity and diversity, unlocking a vast space of reproducible styles from minimal input. Extensive experiments validate that CoTyle effectively turns a numerical code into a style controller, demonstrating a style is worth one code.