Performing Structured Improvisations with pre-trained Deep Learning Models

TL;DR

This paper introduces a system that integrates pre-trained deep generative models into live music performances, addressing challenges of timing, harmony, and style to enhance improvisation.

Contribution

It presents a novel approach to incorporate off-the-shelf deep learning models into live music, leveraging musician expertise to overcome style and timing limitations.

Findings

Enables real-time structured improvisation with deep models

Addresses synchronization and style adaptation in live settings

Facilitates practical use of generative models in performances

Abstract

The quality of outputs produced by deep generative models for music have seen a dramatic improvement in the last few years. However, most deep learning models perform in "offline" mode, with few restrictions on the processing time. Integrating these types of models into a live structured performance poses a challenge because of the necessity to respect the beat and harmony. Further, these deep models tend to be agnostic to the style of a performer, which often renders them impractical for live performance. In this paper we propose a system which enables the integration of out-of-the-box generative models by leveraging the musician's creativity and expertise.