Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation

TL;DR

Spectral Evolution Search (SES) is a novel, efficient method for optimizing initial noise in generative models by focusing on low-frequency components, improving quality and reducing computational costs without parameter updates.

Contribution

SES introduces a spectral bias-aware, gradient-free optimization framework that enhances inference-time scaling for reward-aligned image generation.

Findings

SES outperforms baseline methods in quality-cost trade-offs

Spectral Scaling Prediction explains perturbation impacts across frequencies

SES achieves significant efficiency improvements in experiments

Abstract

Inference-time scaling offers a versatile paradigm for aligning visual generative models with downstream objectives without parameter updates. However, existing approaches that optimize the high-dimensional initial noise suffer from severe inefficiency, as many search directions exert negligible influence on the final generation. We show that this inefficiency is closely related to a spectral bias in generative dynamics: model sensitivity to initial perturbations diminishes rapidly as frequency increases. Building on this insight, we propose Spectral Evolution Search (SES), a plug-and-play framework for initial noise optimization that executes gradient-free evolutionary search within a low-frequency subspace. Theoretically, we derive the Spectral Scaling Prediction from perturbation propagation dynamics, which explains the systematic differences in the impact of perturbations across frequencies. Extensive experiments demonstrate that SES significantly advances the Pareto frontier of generation quality versus computational cost, consistently outperforming strong baselines under equivalent budgets.