MOVA: Towards Scalable and Synchronized Video-Audio Generation

TL;DR

MOVA is an open-source, scalable model that generates synchronized audio-visual content, including speech, sound effects, and music, using a Mixture-of-Experts architecture to advance research and creative applications.

Contribution

The paper introduces MOVA, a novel open-source model with a large MoE architecture supporting synchronized audio-visual generation, addressing limitations of previous closed-source systems.

Findings

Supports high-quality, synchronized lip-synced speech and sound effects

Employs a 32B parameter MoE architecture with 18B active during inference

Provides comprehensive tools for efficient inference and fine-tuning

Abstract

Audio is indispensable for real-world video, yet generation models have largely overlooked audio components. Current approaches to producing audio-visual content often rely on cascaded pipelines, which increase cost, accumulate errors, and degrade overall quality. While systems such as Veo 3 and Sora 2 emphasize the value of simultaneous generation, joint multimodal modeling introduces unique challenges in architecture, data, and training. Moreover, the closed-source nature of existing systems limits progress in the field. In this work, we introduce MOVA (MOSS Video and Audio), an open-source model capable of generating high-quality, synchronized audio-visual content, including realistic lip-synced speech, environment-aware sound effects, and content-aligned music. MOVA employs a Mixture-of-Experts (MoE) architecture, with a total of 32B parameters, of which 18B are active during inference. It supports IT2VA (Image-Text to Video-Audio) generation task. By releasing the model weights and code, we aim to advance research and foster a vibrant community of creators. The released codebase features comprehensive support for efficient inference, LoRA fine-tuning, and prompt enhancement.