Live Music Models

TL;DR

This paper presents Magenta RealTime and Lyria RealTime, innovative live music generation models enabling real-time, user-controlled music creation with superior quality and interactive capabilities, advancing AI-assisted live music performance.

Contribution

Introduction of live music models that generate continuous, real-time music with synchronized user control, outperforming existing models in quality and offering new live performance features.

Findings

Magenta RealTime outperforms other open-weights models on quality metrics.

Models enable real-time, live music generation with user control.

Lyria RealTime provides extensive control and wide prompt coverage.

Abstract

We introduce a new class of generative models for music called live music models that produce a continuous stream of music in real-time with synchronized user control. We release Magenta RealTime, an open-weights live music model that can be steered using text or audio prompts to control acoustic style. On automatic metrics of music quality, Magenta RealTime outperforms other open-weights music generation models, despite using fewer parameters and offering first-of-its-kind live generation capabilities. We also release Lyria RealTime, an API-based model with extended controls, offering access to our most powerful model with wide prompt coverage. These models demonstrate a new paradigm for AI-assisted music creation that emphasizes human-in-the-loop interaction for live music performance.