Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise
Tornike Karchkhadze, Keren Shao, Shlomo Dubnov

TL;DR
This paper introduces a novel AI-based approach that interprets graphic notation from Cornelius Cardew's Treatise, converting visual scores into music through descriptive prompts and diffusion models, enabling new experimental composition methods.
Contribution
It presents a new method combining AI interpretation of graphic scores with diffusion models for music generation, including a novel outpainting technique for seamless compositions.
Findings
AI can effectively interpret graphic notation into musical prompts.
The diffusion model generates cohesive and expressive musical pieces.
The approach expands creative possibilities in experimental music composition.
Abstract
This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression. By leveraging OpenAI's ChatGPT to interpret the abstract visual elements of Treatise, we convert these graphical images into descriptive textual prompts. These prompts are then input into MusicLDM, a pre-trained latent diffusion model designed for music generation. We introduce a technique called "outpainting," which overlaps sections of AI-generated music to create a seamless and cohesive composition. We demostrate a new perspective on performing and interpreting graphic scores, showing how AI can transform visual stimuli into sound and expand the creative possibilities in contemporary/experimental music composition. Musical pieces are available at https://bit.ly/TreatiseAI
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic and Audio Processing · Music Technology and Sound Studies · Neuroscience and Music Perception
MethodsDiffusion · Latent Diffusion Model
