Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise

TL;DR

This paper introduces a novel AI-based approach that interprets graphic notation from Cornelius Cardew's Treatise, converting visual scores into music through descriptive prompts and diffusion models, enabling new experimental composition methods.

Contribution

It presents a new method combining AI interpretation of graphic scores with diffusion models for music generation, including a novel outpainting technique for seamless compositions.

Findings

AI can effectively interpret graphic notation into musical prompts.

The diffusion model generates cohesive and expressive musical pieces.

The approach expands creative possibilities in experimental music composition.

Abstract

This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression. By leveraging OpenAI's ChatGPT to interpret the abstract visual elements of Treatise, we convert these graphical images into descriptive textual prompts. These prompts are then input into MusicLDM, a pre-trained latent diffusion model designed for music generation. We introduce a technique called "outpainting," which overlaps sections of AI-generated music to create a seamless and cohesive composition. We demostrate a new perspective on performing and interpreting graphic scores, showing how AI can transform visual stimuli into sound and expand the creative possibilities in contemporary/experimental music composition. Musical pieces are available at https://bit.ly/TreatiseAI