A Hierarchical Attention Based Seq2seq Model for Chinese Lyrics Generation

TL;DR

This paper introduces a hierarchical attention-based Seq2Seq model for generating Chinese song lyrics that maintains topic relevance and consistency by encoding word and sentence-level context.

Contribution

It proposes a novel hierarchical attention mechanism in Seq2Seq models specifically designed for Chinese lyrics generation, leveraging a large corpus for training.

Findings

Model produces more coherent and topic-consistent lyrics.

Automatic and human evaluations confirm improved quality.

Successfully generates complete lyrics with unified topic constraint.

Abstract

In this paper, we comprehensively study on context-aware generation of Chinese song lyrics. Conventional text generative models generate a sequence or sentence word by word, failing to consider the contextual relationship between sentences. Taking account into the characteristics of lyrics, a hierarchical attention based Seq2Seq (Sequence-to-Sequence) model is proposed for Chinese lyrics generation. With encoding of word-level and sentence-level contextual information, this model promotes the topic relevance and consistency of generation. A large Chinese lyrics corpus is also leveraged for model training. Eventually, results of automatic and human evaluations demonstrate that our model is able to compose complete Chinese lyrics with one united topic constraint.