MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences

TL;DR

This paper introduces MVReward, a reward model trained on human preferences to evaluate and improve multi-view diffusion models for 3D content generation, addressing evaluation fairness and alignment issues.

Contribution

The paper presents a new human-aligned reward model and a multi-view preference learning strategy to improve evaluation and tuning of multi-view diffusion models.

Findings

MVReward effectively encodes human preferences for multi-view assets.

The proposed MVP method improves alignment of diffusion models with human judgments.

Extensive experiments validate the reliability of MVReward as an evaluation metric.

Abstract

Recent years have witnessed remarkable progress in 3D content generation. However, corresponding evaluation methods struggle to keep pace. Automatic approaches have proven challenging to align with human preferences, and the mixed comparison of text- and image-driven methods often leads to unfair evaluations. In this paper, we present a comprehensive framework to better align and evaluate multi-view diffusion models with human preferences. To begin with, we first collect and filter a standardized image prompt set from DALL$\cdot$E and Objaverse, which we then use to generate multi-view assets with several multi-view diffusion models. Through a systematic ranking pipeline on these assets, we obtain a human annotation dataset with 16k expert pairwise comparisons and train a reward model, coined MVReward, to effectively encode human preferences. With MVReward, image-driven 3D methods can be evaluated against each other in a more fair and transparent manner. Building on this, we further propose Multi-View Preference Learning (MVP), a plug-and-play multi-view diffusion tuning strategy. Extensive experiments demonstrate that MVReward can serve as a reliable metric and MVP consistently enhances the alignment of multi-view diffusion models with human preferences.