PromptEvolver: Prompt Inversion through Evolutionary Optimization in Natural-Language Space

TL;DR

PromptEvolver introduces an evolutionary algorithm approach for prompt inversion in text-to-image models, achieving high-fidelity reconstructions with natural language prompts while working in a black-box setting.

Contribution

It is the first to apply genetic algorithms for prompt inversion, enabling high-quality, interpretable prompts without access to model internals.

Findings

Outperforms existing prompt inversion methods across benchmarks.

Generates more natural and faithful prompts for target images.

Works effectively on black-box models using only image outputs.

Abstract

Text-to-image generation has progressed rapidly, but faithfully generating complex scenes requires extensive trial-and-error to find the exact prompt. In the prompt inversion task, the goal is to recover a textual prompt that can faithfully reconstruct a given target image. Currently, existing methods frequently yield suboptimal reconstructions and produce unnatural, hard-to-interpret prompts that hinder transparency and controllability. In this work, we present PromptEvolver, a prompt inversion approach that generates natural-language prompts while achieving high-fidelity reconstructions of the target image. Our method uses a genetic algorithm to optimize the prompt, leveraging a strong vision-language model to guide the evolution process. Importantly, it works on black-box generation models by requiring only image outputs. Finally, we evaluate PromptEvolver across multiple prompt inversion benchmarks and show that it consistently outperforms competing methods.