Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models
Niklas Schweiger, Daniel Cremers, Karnik Ram

TL;DR
This paper introduces TRS, a trust-region based black-box optimization method for aligning diffusion and flow models to target rewards, improving output quality across various generative tasks.
Contribution
The paper presents a simple, versatile trust-region search algorithm that optimizes source noise in black-box models, outperforming existing methods in alignment tasks.
Findings
Significantly improved sample quality over base models.
Effective across text-to-image, molecule, and protein design.
Minimal hyperparameter tuning required.
Abstract
Optimizing the noise samples of diffusion and flow models is an increasingly popular approach to align these models to target rewards at inference time. However, we observe that these approaches are usually restricted to differentiable or cheap reward models, the formulation of the underlying pretrained generative model, or are memory/compute inefficient. We instead propose a simple trust-region based search algorithm (TRS) which treats the pre-trained generative and reward models as a black-box and only optimizes the source noise. Our approach achieves a good balance between global exploration and local exploitation, and is versatile and easily adaptable to various generative settings and reward models with minimal hyperparameter tuning. We evaluate TRS across text-to-image, molecule and protein design tasks, and obtain significantly improved output samples over the base generative…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Model Reduction and Neural Networks · Protein Structure and Dynamics
