RoboJam: A Musical Mixture Density Network for Collaborative Touchscreen Interaction

TL;DR

RoboJam is a neural network-based system that generates responsive touchscreen interactions to assist users in a music app, using a mixture density network to predict touch locations and timings.

Contribution

The paper introduces RoboJam, a novel neural network architecture employing a mixture density layer for generating realistic touchscreen responses in a musical context.

Findings

Effective generation of touchscreen interactions

Successful integration into a music app

Preliminary positive evaluation results

Abstract

RoboJam is a machine-learning system for generating music that assists users of a touchscreen music app by performing responses to their short improvisations. This system uses a recurrent artificial neural network to generate sequences of touchscreen interactions and absolute timings, rather than high-level musical notes. To accomplish this, RoboJam's network uses a mixture density layer to predict appropriate touch interaction locations in space and time. In this paper, we describe the design and implementation of RoboJam's network and how it has been integrated into a touchscreen music app. A preliminary evaluation analyses the system in terms of training, musical generation and user interaction.