WakaVT: A Sequential Variational Transformer for Waka Generation

TL;DR

WakaVT is a novel model combining Transformer and variational autoencoder techniques to generate high-quality Japanese Waka poetry from user keywords, addressing form, diversity, and linguistic quality.

Contribution

This paper introduces the first Transformer-variational autoencoder model for Waka generation, improving form adherence, diversity, and linguistic quality in Japanese poetry AI.

Findings

WakaVT outperforms baseline models in objective metrics.

Subjective evaluations favor WakaVT's poetic quality.

The model effectively balances form, diversity, and fluency.

Abstract

Poetry generation has long been a challenge for artificial intelligence. In the scope of Japanese poetry generation, many researchers have paid attention to Haiku generation, but few have focused on Waka generation. To further explore the creative potential of natural language generation systems in Japanese poetry creation, we propose a novel Waka generation model, WakaVT, which automatically produces Waka poems given user-specified keywords. Firstly, an additive mask-based approach is presented to satisfy the form constraint. Secondly, the structures of Transformer and variational autoencoder are integrated to enhance the quality of generated content. Specifically, to obtain novelty and diversity, WakaVT employs a sequence of latent variables, which effectively captures word-level variability in Waka data. To improve linguistic quality in terms of fluency, coherence, and meaningfulness, we further propose the fused multilevel self-attention mechanism, which properly models the hierarchical linguistic structure of Waka. To the best of our knowledge, we are the first to investigate Waka generation with models based on Transformer and/or variational autoencoder. Both objective and subjective evaluation results demonstrate that our model outperforms baselines significantly.