Boosting Text-to-Image Diffusion Models via Core Token Attention-Based Seed Selection

TL;DR

This paper introduces ABSS, a seed selection method based on attention dynamics over prompt core tokens, which improves image quality and alignment in text-to-image diffusion models without additional training.

Contribution

The paper proposes a novel, training-free seed ranking technique using attention over core tokens, enhancing existing diffusion models' performance at inference time.

Findings

ABSS improves text-image alignment and visual quality across benchmarks.

It requires no fine-tuning or changes to initial noise.

ABSS consistently outperforms baseline seed selection methods.

Abstract

Text-to-image diffusion models can synthesize high-quality images, yet the outcome is notoriously sensitive to the random seed: different initial seeds often yield large variations in image quality and prompt-image alignment. We revisit this "seed effect" and show that attention dynamics over prompt core tokens, the content-bearing words, measured during the first few denoising steps, strongly predict final generation quality. Building on this observation, we introduce Attention-Based Seed Selection (ABSS), a training-free, plug-and-play method that ranks seeds for a given prompt by leveraging cross-attention to core tokens during the denoising process. ABSS requires no finetuning and does not alter the initial noise; it scores and ranks all candidate seeds, keeps only the top-k for full generation, and discards the rest, without relying on a fixed accept/reject threshold. Operating purely at inference time, ABSS can serve as a lightweight pre-selection add-on for existing seed-optimization pipelines, enabling additional gains. Across three benchmarks, extensive experiments show that ABSS enables consistent improvements in text-image alignment and visual quality for Stable Diffusion variants, as corroborated by human preference and alignment metrics.