Piano Genie

TL;DR

Piano Genie is an AI-powered controller enabling non-musicians to improvise on the piano using a simple button interface, translating limited input into plausible piano music in real time.

Contribution

It introduces a novel neural network autoencoder approach with discrete bottlenecks for real-time music generation from minimal input.

Findings

Real-time plausible piano music generation from 8-button input

Effective mapping learned via recurrent neural network autoencoders

Enhanced user experience through musically meaningful constraints

Abstract

We present Piano Genie, an intelligent controller which allows non-musicians to improvise on the piano. With Piano Genie, a user performs on a simple interface with eight buttons, and their performance is decoded into the space of plausible piano music in real time. To learn a suitable mapping procedure for this problem, we train recurrent neural network autoencoders with discrete bottlenecks: an encoder learns an appropriate sequence of buttons corresponding to a piano piece, and a decoder learns to map this sequence back to the original piece. During performance, we substitute a user's input for the encoder output, and play the decoder's prediction each time the user presses a button. To improve the intuitiveness of Piano Genie's performance behavior, we impose musically meaningful constraints over the encoder's outputs.