Preference Adaptive and Sequential Text-to-Image Generation

TL;DR

This paper introduces PASTA, an RL-based system that iteratively improves text-to-image generation through user preferences, enabling adaptive, multi-turn, collaborative image creation with demonstrated human-rated improvements.

Contribution

It presents a novel RL agent for sequential T2I generation that adapts to user preferences and introduces a new dataset of human preference sequences for training and evaluation.

Findings

PASTA significantly outperforms baseline methods in human evaluations.

The dataset of sequential preferences enables better modeling of user choices.

Adaptive prompt expansion improves collaborative image creation.

Abstract

We address the problem of interactive text-to-image (T2I) generation, designing a reinforcement learning (RL) agent which iteratively improves a set of generated images for a user through a sequence of prompt expansions. Using human raters, we create a novel dataset of sequential preferences, which we leverage, together with large-scale open-source (non-sequential) datasets. We construct user-preference and user-choice models using an EM strategy and identify varying user preference types. We then leverage a large multimodal language model (LMM) and a value-based RL approach to suggest an adaptive and diverse slate of prompt expansions to the user. Our Preference Adaptive and Sequential Text-to-image Agent (PASTA) extends T2I models with adaptive multi-turn capabilities, fostering collaborative co-creation and addressing uncertainty or underspecification in a user's intent. We evaluate PASTA using human raters, showing significant improvement compared to baseline methods. We also open-source our sequential rater dataset and simulated user-rater interactions to support future research in user-centric multi-turn T2I systems.