Deep Attention-Based Alignment Network for Melody Generation from Incomplete Lyrics

TL;DR

This paper introduces a deep attention-based neural network that aligns incomplete lyrics with melodies, enabling automatic lyric and melody generation for music composition.

Contribution

It presents a novel encoder-decoder model with attention mechanism for lyrics-to-melody alignment, trained to generate song pairs from incomplete lyrics.

Findings

Effective in generating proper lyrics and melodies from incomplete inputs

Attention mechanism improves alignment accuracy

Qualitative and quantitative evaluations confirm method's capability

Abstract

We propose a deep attention-based alignment network, which aims to automatically predict lyrics and melody with given incomplete lyrics as input in a way similar to the music creation of humans. Most importantly, a deep neural lyrics-to-melody net is trained in an encoder-decoder way to predict possible pairs of lyrics-melody when given incomplete lyrics (few keywords). The attention mechanism is exploited to align the predicted lyrics with the melody during the lyrics-to-melody generation. The qualitative and quantitative evaluation metrics reveal that the proposed method is indeed capable of generating proper lyrics and corresponding melody for composing new songs given a piece of incomplete seed lyrics.