Instant Preference Alignment for Text-to-Image Diffusion Models

TL;DR

This paper introduces a training-free, real-time framework for preference-aligned text-to-image generation using multimodal large language models to understand preferences and guide diffusion models, enabling interactive and nuanced image creation.

Contribution

The proposed method is the first to achieve instant, preference-aligned T2I generation without training, leveraging MLLMs for preference understanding and control, supporting multi-round refinement.

Findings

Outperforms prior methods in quantitative metrics

Achieves superior human evaluation scores

Enables real-time, interactive image generation

Abstract

Text-to-image (T2I) generation has greatly enhanced creative expression, yet achieving preference-aligned generation in a real-time and training-free manner remains challenging. Previous methods often rely on static, pre-collected preferences or fine-tuning, limiting adaptability to evolving and nuanced user intents. In this paper, we highlight the need for instant preference-aligned T2I generation and propose a training-free framework grounded in multimodal large language model (MLLM) priors. Our framework decouples the task into two components: preference understanding and preference-guided generation. For preference understanding, we leverage MLLMs to automatically extract global preference signals from a reference image and enrich a given prompt using structured instruction design. Our approach supports broader and more fine-grained coverage of user preferences than existing methods. For preference-guided generation, we integrate global keyword-based control and local region-aware cross-attention modulation to steer the diffusion model without additional training, enabling precise alignment across both global attributes and local elements. The entire framework supports multi-round interactive refinement, facilitating real-time and context-aware image generation. Extensive experiments on the Viper dataset and our collected benchmark demonstrate that our method outperforms prior approaches in both quantitative metrics and human evaluations, and opens up new possibilities for dialog-based generation and MLLM-diffusion integration.