A Design Space for Live Music Agents

TL;DR

This paper creates a comprehensive design space for live music agents, categorizing existing systems and identifying trends and gaps to guide future research and development in real-time AI-supported music performance.

Contribution

It synthesizes diverse perspectives to map the landscape of live music agents and introduces a structured design space for future exploration.

Findings

Mapped 184 live music systems from literature and video.

Identified key trends and gaps in live music agent design.

Provided a structured framework for future research and development.

Abstract

Live music provides a uniquely rich setting for studying creativity and interaction due to its spontaneous nature. The pursuit of live music agents--intelligent systems supporting real-time music performance and interaction--has captivated researchers across HCI, AI, and computer music for decades, and recent advancements in AI suggest unprecedented opportunities to evolve their design. However, the interdisciplinary nature of music has led to fragmented development across research communities, hindering effective communication and collaborative progress. In this work, we bring together perspectives from these diverse fields to map the current landscape of live music agents. Based on our analysis of 184 systems across both academic literature and video, we develop a comprehensive design space that categorizes dimensions spanning usage contexts, interactions, technologies, and ecosystems. By highlighting trends and gaps in live music agents, our design space offers researchers, designers, and musicians a structured lens to understand existing systems and shape future directions in real-time human-AI music co-creation. We release our annotated systems as a living artifact at https://live-music-agents.github.io.