Theme-aware generation model for chinese lyrics

TL;DR

This paper introduces a theme-aware Chinese lyrics generation model that enhances thematic coherence and fluency using a multi-channel seq2seq architecture with attention and LDA-based theme extraction.

Contribution

It presents a novel multi-channel seq2seq model incorporating theme information and attention mechanisms, improving coherence in Chinese lyric generation.

Findings

Generated lyrics are grammatically correct and semantically coherent.

The model effectively fuses theme and context for improved generation.

Applicable to other natural language generation tasks.

Abstract

With rapid development of neural networks, deep-learning has been extended to various natural language generation fields, such as machine translation, dialogue generation and even literature creation. In this paper, we propose a theme-aware language generation model for Chinese music lyrics, which improves the theme-connectivity and coherence of generated paragraphs greatly. A multi-channel sequence-to-sequence (seq2seq) model encodes themes and previous sentences as global and local contextual information. Moreover, attention mechanism is incorporated for sequence decoding, enabling to fuse context into predicted next texts. To prepare appropriate train corpus, LDA (Latent Dirichlet Allocation) is applied for theme extraction. Generated lyrics is grammatically correct and semantically coherent with selected themes, which offers a valuable modelling method in other fields including multi-turn chatbots, long paragraph generation and etc.