ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinement
Keshav Bhandari, Sungkyun Chang, Tongyu Lu, Fareza R. Enus, Louis B. Bradshaw, Dorien Herremans, Simon Colton

TL;DR
ImprovNet is a transformer-based model that enables controllable, expressive musical improvisations and style transfer across genres, using an iterative corruption-refinement approach to modify melody, harmony, or rhythm while maintaining musical coherence.
Contribution
It introduces a unified transformer architecture with a self-supervised training strategy for controllable, multi-task musical style transfer and improvisation, outperforming previous models in coherence and genre recognition.
Findings
Outperforms Anticipatory Music Transformer in short tasks
Achieves 79% genre recognition accuracy in style transfer
Effectively controls style transfer degree and structural similarity
Abstract
Despite deep learning's remarkable advances in style transfer across various domains, generating controllable performance-level musical style transfer for complete symbolically represented musical works remains a challenging area of research. Much of this is owed to limited datasets, especially for genres such as jazz, and the lack of unified models that can handle multiple music generation tasks. This paper presents ImprovNet, a transformer-based architecture that generates expressive and controllable musical improvisations through a self-supervised corruption-refinement training strategy. The improvisational style transfer is aimed at making meaningful modifications to one or more musical elements - melody, harmony or rhythm of the original composition with respect to the target genre. ImprovNet unifies multiple capabilities within a single model: it can perform cross-genre and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic Technology and Sound Studies · Music and Audio Processing · Neuroscience and Music Perception
MethodsAttention Is All You Need · Label Smoothing · Byte Pair Encoding · Layer Normalization · Residual Connection · Dense Connections · Linear Layer · Multi-Head Attention · Position-Wise Feed-Forward Layer · Adam
