Song From PI: A Musically Plausible Network for Pop Music Generation

TL;DR

This paper introduces a hierarchical RNN framework for pop music generation that encodes musical structure, producing more preferred compositions and enabling applications like neural dancing, karaoke, and story singing.

Contribution

It presents a novel hierarchical RNN model that incorporates prior musical knowledge for more plausible pop music synthesis.

Findings

Generated music is preferred over recent Google method in human studies.

Framework enables applications like neural dancing and karaoke.

Model effectively captures hierarchical musical structure.

Abstract

We present a novel framework for generating pop music. Our model is a hierarchical Recurrent Neural Network, where the layers and the structure of the hierarchy encode our prior knowledge about how pop music is composed. In particular, the bottom layers generate the melody, while the higher levels produce the drums and chords. We conduct several human studies that show strong preference of our generated music over that produced by the recent method by Google. We additionally show two applications of our framework: neural dancing and karaoke, as well as neural story singing.