Steering Large Text-to-Image Model for Abstract Art Synthesis: Preference-based Prompt Optimization and Visualization

TL;DR

This paper presents a novel, user-guided method for generating abstract art using large text-to-image models, combining a deterministic Artist Model with real-time human feedback to optimize prompts and produce personalized artwork.

Contribution

It introduces a prompt-free generative approach that integrates genetic algorithms and human feedback to customize abstract art creation with large text-to-image models.

Findings

Effective prompt optimization through human feedback

Successful generation of style-specific abstract art

Enhanced user satisfaction with personalized outputs

Abstract

With the advancement of neural generative capabilities, the art community has increasingly embraced GenAI (Generative Artificial Intelligence), particularly large text-to-image models, for producing aesthetically compelling results. However, the process often lacks determinism and requires a tedious trial-and-error process as users often struggle to devise effective prompts to achieve their desired outcomes. This paper introduces a prompting-free generative approach that applies a genetic algorithm and real-time iterative human feedback to optimize prompt generation, enabling the creation of user-preferred abstract art through a customized Artist Model. The proposed two-part approach begins with constructing an Artist Model capable of deterministically generating abstract art in specific styles, e.g., Kandinsky's Bauhaus style. The second phase integrates real-time user feedback to optimize the prompt generation and obtains an Optimized Prompting Model, which adapts to user preferences and generates prompts automatically. When combined with the Artist Model, this approach allows users to create abstract art tailored to their personal preferences and artistic style.