Opening the Design Space: Two Years of Performance with Intelligent Musical Instruments

TL;DR

This paper presents a portable, affordable AI-based musical instrument platform that enables artists to experiment with generative AI in musical creation, demonstrating five innovative instrument designs over two years.

Contribution

It introduces a low-cost, artist-centered AI instrument platform and provides five example instruments developed through a two-year artistic research process.

Findings

Re mapping can replace retraining for AI interaction discovery.

Fast input interleaving offers a new co-creative strategy.

Small-data AI models are transportable design resources.

Abstract

Machine generation of symbolic music and digital audio are hot topics but there have been relatively few digital musical instruments that integrate generative AI. Present musical AI tools are not artist centred and do not support experimentation or integrating into musical instruments or practices. This work introduces an inexpensive generative AI instrument platform based on a single board computer that connects via MIDI to other musical devices. The platform uses artist-collected datasets with models trained on a regular computer. This paper asks what the design space of intelligent musical instruments might look like when accessible and portable AI systems are available for artistic exploration. I contribute five examples of instruments created and tested through a two-year first-person artistic research process. These show that (re)mapping can replace retraining for discovering AI interaction, that fast input interleaving is a new co-creative strategy, that small-data AI models can be a transportable design resource, and that cheap hardware can lower barriers to inclusion. This work could enable artists to explore new interaction and performance schemes with intelligent musical instruments.