V2A-DPO: Omni-Preference Optimization for Video-to-Audio Generation

TL;DR

This paper presents V2A-DPO, a new framework for optimizing video-to-audio generation models to better align with human preferences, achieving state-of-the-art results on benchmark datasets.

Contribution

The paper introduces a novel DPO framework with a comprehensive scoring system, automated preference data generation, and curriculum learning for flow-based V2A models.

Findings

V2A-DPO outperforms Denoising Diffusion Policy Optimization (DDPO) and pre-trained baselines.

State-of-the-art performance achieved on benchmark VGGSound dataset.

Improved alignment of generated audio with human preferences.

Abstract

This paper introduces V2A-DPO, a novel Direct Preference Optimization (DPO) framework tailored for flow-based video-to-audio generation (V2A) models, incorporating key adaptations to effectively align generated audio with human preferences. Our approach incorporates three core innovations: (1) AudioScore-a comprehensive human preference-aligned scoring system for assessing semantic consistency, temporal alignment, and perceptual quality of synthesized audio; (2) an automated AudioScore-driven pipeline for generating large-scale preference pair data for DPO optimization; (3) a curriculum learning-empowered DPO optimization strategy specifically tailored for flow-based generative models. Experiments on benchmark VGGSound dataset demonstrate that human-preference aligned Frieren and MMAudio using V2A-DPO outperform their counterparts optimized using Denoising Diffusion Policy Optimization (DDPO) as well as pre-trained baselines. Furthermore, our DPO-optimized MMAudio achieves state-of-the-art performance across multiple metrics, surpassing published V2A models.