HyperAlign: Hypernetwork for Efficient Test-Time Alignment of Diffusion Models

TL;DR

HyperAlign introduces a hypernetwork-based framework that enables efficient, test-time alignment of diffusion models, improving semantic consistency and visual quality without high computational costs.

Contribution

The paper presents HyperAlign, a novel hypernetwork approach for test-time diffusion model alignment that balances quality and efficiency, addressing limitations of existing methods.

Findings

HyperAlign outperforms existing methods in semantic consistency.

HyperAlign improves visual quality across multiple generative models.

The framework balances alignment effectiveness with computational efficiency.

Abstract

Diffusion model alignment aims to bridge the gap between generated outputs and human preferences by enhancing both semantic consistency with textual prompts and overall visual quality. Existing alignment methods face a challenging trade-off: test-time approaches enable input-specific adaptability but introduce significant computational overhead and tend to under-optimize, while fine-tuning approaches risk reward over-optimization and loss of generation diversity. To bridge this gap, we propose HyperAlign, a framework that trains a hypernetwork for efficient and effective test-time alignment. Instead of modifying latent states directly, HyperAlign dynamically generates input-and-state-conditioned low-rank adaptation weights to modulate the denoising trajectory toward target rewards. We introduce multiple HyperAlign variants of varying granularity to balance alignment quality and computational efficiency. The hypernetwork is optimized with a reward objective regularized by preference data to mitigate reward hacking. We evaluate HyperAlign across multiple generative paradigms, including Stable Diffusion and FLUX, where it significantly outperforms existing alignment methods in semantic consistency and visual quality.