TL;DR
This paper compares various text generation models' ability to produce poetry in the style of early English Romanticism, evaluating their quality and coherence with automatic metrics.
Contribution
It introduces a comparative analysis of different neural network models for creative poetry generation, highlighting the performance differences between RNNs and transformer-based models.
Findings
Transformer models outperform RNNs in poetry quality.
Larger transformer models generate more coherent and higher-quality poems.
This is among the first comparisons of these models in a creative context.
Abstract
In this paper we compare various text generation models' ability to write poetry in the style of early English Romanticism. These models include: Character-Level Recurrent Neural Networks with Long Short-Term Memory, Hugging Face's GPT-2, OpenAI's GPT-3, and EleutherAI's GPT-NEO. Quality was measured based syllable count and coherence with the automatic evaluation metric GRUEN. Character-Level Recurrent Neural Networks performed far worse compared to transformer models. And, as parameter-size increased, the quality of transformer models' poems improved. These models are typically not compared in a creative context, and we are happy to contribute.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Text Analysis Techniques · Topic Modeling · Neural Networks and Applications
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Linear Layer · Cosine Annealing · Discriminative Fine-Tuning · Attention Dropout · Dropout · Multi-Head Attention · Dense Connections · Softmax