MagicWand: A Universal Agent for Generation and Evaluation Aligned with User Preference

TL;DR

MagicWand is a versatile AI agent that improves content generation and evaluation by aligning with user preferences, supported by a large dataset and benchmark for diverse AIGC tasks.

Contribution

We introduce MagicWand, a universal agent for generation and evaluation aligned with user preferences, along with UniPrefer-100K dataset and UniPreferBench benchmark.

Findings

MagicWand outperforms baselines in preference alignment.

The dataset and benchmark facilitate better evaluation of preference-aware AIGC.

Experiments show consistent improvement across diverse tasks.

Abstract

Recent advances in AIGC (Artificial Intelligence Generated Content) models have enabled significant progress in image and video generation. However, users still struggle to obtain content that aligns with their preferences due to the difficulty of crafting detailed prompts and the lack of mechanisms to retain their preferences. To address these challenges, we construct \textbf{UniPrefer-100K}, a large-scale dataset comprising images, videos, and associated text that describes the styles users tend to prefer. Based on UniPrefer-100K, we propose \textbf{MagicWand}, a universal generation and evaluation agent that enhances prompts based on user preferences, leverages advanced generation models for high-quality content, and applies preference-aligned evaluation and refinement. In addition, we introduce \textbf{UniPreferBench}, the first large-scale benchmark with over 120K annotations for assessing user preference alignment across diverse AIGC tasks. Experiments on UniPreferBench demonstrate that MagicWand consistently generates content and evaluations that are well aligned with user preferences across a wide range of scenarios.