RewardFlow: Generate Images by Optimizing What You Reward

TL;DR

RewardFlow is a novel inference-time framework that optimizes multiple rewards to generate semantically aligned and high-fidelity images using pretrained diffusion models.

Contribution

It introduces a unified, inversion-free approach with a prompt-aware adaptive policy for multi-reward optimization in image generation.

Findings

Achieves state-of-the-art edit fidelity on image editing benchmarks.

Demonstrates improved compositional alignment in generated images.

Effectively integrates diverse rewards including language-vision reasoning.

Abstract

We introduce RewardFlow, an inversion-free framework that steers pretrained diffusion and flow-matching models at inference time through multi-reward Langevin dynamics. RewardFlow unifies complementary differentiable rewards for semantic alignment, perceptual fidelity, localized grounding, object consistency, and human preference, and further introduces a differentiable VQA-based reward that provides fine-grained semantic supervision through language-vision reasoning. To coordinate these heterogeneous objectives, we design a prompt-aware adaptive policy that extracts semantic primitives from the instruction, infers edit intent, and dynamically modulates reward weights and step sizes throughout sampling. Across several image editing and compositional generation benchmarks, RewardFlow delivers state-of-the-art edit fidelity and compositional alignment.