Na\"ive PAINE: Lightweight Text-to-Image Generation Improvement with Prompt Evaluation

TL;DR

Na"ive PAINE enhances text-to-image diffusion models by predicting initial noise quality and selecting optimal seeds, leading to improved image quality without extensive multiple generations.

Contribution

It introduces a lightweight method that predicts image quality from initial noise and prompt, optimizing diffusion model outputs efficiently.

Findings

Outperforms existing methods on prompt benchmarks

Improves image quality with fewer generation attempts

Seamlessly integrates into existing pipelines

Abstract

Text-to-Image (T2I) generation is primarily driven by Diffusion Models (DM) which rely on random Gaussian noise. Thus, like playing the slots at a casino, a DM will produce different results given the same user-defined inputs. This imposes a gambler's burden: To perform multiple generation cycles to obtain a satisfactory result. However, even though DMs use stochastic sampling to seed generation, the distribution of generated content quality highly depends on the prompt and the generative ability of a DM with respect to it. To account for this, we propose Na\"ive PAINE for improving the generative quality of Diffusion Models by leveraging T2I preference benchmarks. We directly predict the numerical quality of an image from the initial noise and given prompt. Na\"ive PAINE then selects a handful of quality noises and forwards them to the DM for generation. Further, Na\"ive PAINE provides feedback on the DM generative quality given the prompt and is lightweight enough to seamlessly fit into existing DM pipelines. Experimental results demonstrate that Na\"ive PAINE outperforms existing approaches on several prompt corpus benchmarks.