Redefining <Creative> in Dictionary: Towards an Enhanced Semantic Understanding of Creative Generation

TL;DR

This paper introduces CreTok, a novel token that enhances diffusion models' understanding of creativity, enabling direct generation of combinatorial concepts without retraining, and demonstrating superior performance in creative image synthesis.

Contribution

CreTok redefines 'creative' as a new token, improving semantic understanding and enabling combinatorial creativity in diffusion models without additional training.

Findings

Achieves state-of-the-art creative generation performance.

Improves text-image alignment in diffusion models.

Higher human preference ratings for generated images.

Abstract

``Creative'' remains an inherently abstract concept for both humans and diffusion models. While text-to-image (T2I) diffusion models can easily generate out-of-distribution concepts like ``a blue banana'', they struggle with generating combinatorial objects such as ``a creative mixture that resembles a lettuce and a mantis'', due to difficulties in understanding the semantic depth of ``creative''. Current methods rely heavily on synthesizing reference prompts or images to achieve a creative effect, typically requiring retraining for each unique creative output-a process that is computationally intensive and limits practical applications. To address this, we introduce CreTok, which brings meta-creativity to diffusion models by redefining ``creative'' as a new token, \texttt{<CreTok>}, thus enhancing models' semantic understanding for combinatorial creativity. CreTok achieves such redefinition by iteratively sampling diverse text pairs from our proposed CangJie dataset to form adaptive prompts and restrictive prompts, and then optimizing the similarity between their respective text embeddings. Extensive experiments demonstrate that <CreTok> enables the universal and direct generation of combinatorial creativity across diverse concepts without additional training, achieving state-of-the-art performance with improved text-image alignment and higher human preference ratings. Code will be made available at https://github.com/fu-feng/CreTok.