Evolve to Inspire: Novelty Search for Diverse Image Generation

TL;DR

WANDER is a novel method that uses evolutionary search guided by language models and CLIP embeddings to generate highly diverse images from a single prompt, addressing the limited diversity of current diffusion models.

Contribution

The paper introduces WANDER, a new approach leveraging LLMs and novelty search to enhance diversity in text-to-image generation, surpassing existing prompt optimization methods.

Findings

WANDER significantly improves image diversity metrics.

Emitters effectively guide the search into diverse prompt regions.

Empirical results outperform baseline evolutionary prompt optimization methods.

Abstract

Text-to-image diffusion models, while proficient at generating high-fidelity images, often suffer from limited output diversity, hindering their application in exploratory and ideation tasks. Existing prompt optimization techniques typically target aesthetic fitness or are ill-suited to the creative visual domain. To address this shortcoming, we introduce WANDER, a novelty search-based approach to generating diverse sets of images from a single input prompt. WANDER operates directly on natural language prompts, employing a Large Language Model (LLM) for semantic evolution of diverse sets of images, and using CLIP embeddings to quantify novelty. We additionally apply emitters to guide the search into distinct regions of the prompt space, and demonstrate that they boost the diversity of the generated images. Empirical evaluations using FLUX-DEV for generation and GPT-4o-mini for mutation demonstrate that WANDER significantly outperforms existing evolutionary prompt optimization baselines in diversity metrics. Ablation studies confirm the efficacy of emitters.