Deep learning and sub-word-unit approach in written art generation

TL;DR

This paper introduces a novel sub-word-unit approach for neural network-based poetry generation, significantly improving the quality of generated poems by better capturing linguistic features, especially in morphologically rich languages like Polish.

Contribution

The work presents a new sub-word-unit method for training RNNs on poetry, including a specialized stemming pre-processing step, which enhances the naturalness and metrical accuracy of generated poems.

Findings

Sub-word units outperform character-based models in poem quality.

The approach effectively captures Polish language morphology.

Generated poems closely follow source text metre and vocabulary.

Abstract

Automatic poetry generation is novel and interesting application of natural language processing research. It became more popular during the last few years due to the rapid development of technology and neural computing power. This line of research can be applied to the study of linguistics and literature, for social science experiments, or simply for entertainment. The most effective known method of artificial poem generation uses recurrent neural networks (RNN). We also used RNNs to generate poems in the style of Adam Mickiewicz. Our network was trained on the Sir Thaddeus poem. For data pre-processing, we used a specialized stemming tool, which is one of the major innovations and contributions of this work. Our experiment was conducted on the source text, divided into sub-word units (at a level of resolution close to syllables). This approach is novel and is not often employed in the published literature. The subwords units seem to be a natural choice for analysis of the Polish language, as the language is morphologically rich due to cases, gender forms and a large vocabulary. Moreover, Sir Thaddeus contains rhymes, so the analysis of syllables can be meaningful. We verified our model with different settings for the temperature parameter, which controls the randomness of the generated text. We also compared our results with similar models trained on the same text but divided into characters (which is the most common approach alongside the use of full word units). The differences were tremendous. Our solution generated much better poems that were able to follow the metre and vocabulary of the source data text.