Composer's Assistant: An Interactive Transformer for Multi-Track MIDI Infilling

TL;DR

Composer's Assistant is an interactive system that uses a transformer model to perform multi-track MIDI infilling within a digital audio workstation, enabling improved human-computer collaborative composition.

Contribution

It introduces a novel transformer-based model and interactive system for multi-track MIDI infilling, integrated into REAPER, with publicly available source code and pretrained models.

Findings

Model achieves effective MIDI infilling in objective tests.

Subjective evaluations favor the system's usability and musical coherence.

System is openly accessible for further research and development.

Abstract

We introduce Composer's Assistant, a system for interactive human-computer composition in the REAPER digital audio workstation. We consider the task of multi-track MIDI infilling when arbitrary track-measures have been deleted from a contiguous slice of measures from a MIDI file, and we train a T5-like model to accomplish this task. Composer's Assistant consists of this model together with scripts that enable interaction with the model in REAPER. We conduct objective and subjective tests of our model. We release our complete system, consisting of source code, pretrained models, and REAPER scripts. Our models were trained only on permissively-licensed MIDI files.