Diverse Melody Generation from Chinese Lyrics via Mutual Information Maximization

TL;DR

This paper introduces DMG, a novel method using mutual information maximization for Chinese lyrics-conditioned melody generation, enhancing diversity, alignment, and overall quality of generated melodies.

Contribution

It adapts mutual information maximization to improve diversity and alignment in Chinese lyrics melody generation, incorporating scheduled sampling and force decoding techniques.

Findings

DMG generates more pleasing melodies than baselines

Improved alignment between lyrics and melodies

Enhanced diversity in generated tunes

Abstract

In this paper, we propose to adapt the method of mutual information maximization into the task of Chinese lyrics conditioned melody generation to improve the generation quality and diversity. We employ scheduled sampling and force decoding techniques to improve the alignment between lyrics and melodies. With our method, which we called Diverse Melody Generation (DMG), a sequence-to-sequence model learns to generate diverse melodies heavily depending on the input style ids, while keeping the tonality and improving the alignment. The experimental results of subjective tests show that DMG can generate more pleasing and coherent tunes than baseline methods.