MusIAC: An extensible generative framework for Music Infilling Applications with multi-level Control

TL;DR

MusIAC is a flexible transformer-based framework for music infilling that incorporates multiple control tokens, allowing for more stylistic and property-specific music generation with an interactive interface.

Contribution

This work introduces an extensible music infilling framework with multi-level control tokens, enhancing stylistic fidelity and user control over generated music.

Findings

Adding control tokens improves stylistic similarity to original music

Inclusion of control tokens allows manipulation of texture and tonal tension

Framework supports interactive music generation via Google Colab

Abstract

We present a novel music generation framework for music infilling, with a user friendly interface. Infilling refers to the task of generating musical sections given the surrounding multi-track music. The proposed transformer-based framework is extensible for new control tokens as the added music control tokens such as tonal tension per bar and track polyphony level in this work. We explore the effects of including several musically meaningful control tokens, and evaluate the results using objective metrics related to pitch and rhythm. Our results demonstrate that adding additional control tokens helps to generate music with stronger stylistic similarities to the original music. It also provides the user with more control to change properties like the music texture and tonal tension in each bar compared to previous research which only provided control for track density. We present the model in a Google Colab notebook to enable interactive generation.