Seed-Music: A Unified Framework for High Quality and Controlled Music Generation

TL;DR

Seed-Music is a versatile music generation framework that combines auto-regressive and diffusion models to produce high-quality, style-controlled vocal music and offers interactive editing tools for post-production refinement.

Contribution

It introduces a unified system supporting both controlled music creation and editing, integrating multiple modalities for style control and editing capabilities.

Findings

Supports multi-modal style control for vocal music

Enables interactive editing of lyrics and melodies

Combines auto-regressive and diffusion models for quality

Abstract

We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio. We encourage readers to listen to demo audio examples at https://team.doubao.com/seed-music "https://team.doubao.com/seed-music".