Interactive Machine Learning of Musical Gesture

TL;DR

This paper reviews Interactive Machine Learning techniques for analyzing and designing musical gestures, highlighting challenges, algorithms, and applications in sound synthesis and musical performance.

Contribution

It introduces Assisted Interactive Machine Learning (AIML) for musical gestures and demonstrates its use in creating four musical pieces, advancing musical practice with IML.

Findings

AIML enables complex gesture-sound interactions

Reinforcement Learning facilitates exploration of musical gestures

IML techniques influence contemporary musical composition

Abstract

This chapter presents an overview of Interactive Machine Learning (IML) techniques applied to the analysis and design of musical gestures. We go through the main challenges and needs related to capturing, analysing, and applying IML techniques to human bodily gestures with the purpose of performing with sound synthesis systems. We discuss how different algorithms may be used to accomplish different tasks, including interacting with complex synthesis techniques and exploring interaction possibilities by means of Reinforcement Learning (RL) in an interaction paradigm we developed called Assisted Interactive Machine Learning (AIML). We conclude the chapter with a description of how some of these techniques were employed by the authors for the development of four musical pieces, thus outlining the implications that IML have for musical practice.