Bits of Grass: Does GPT already know how to write like Whitman?
Piotr Sawicki, Marek Grzes, Fabricio Goes, Dan Brown, Max Peeperkorn,, Aisha Khatun

TL;DR
This paper investigates whether GPT-3.5, GPT-3.5-turbo, and GPT-4 can generate poetry in the style of specific authors using zero-shot and few-shot prompts, finding they do not succeed without fine-tuning even with extensive examples.
Contribution
The study provides an automated evaluation of large language models' ability to imitate specific poetic styles without fine-tuning, highlighting current limitations.
Findings
Models do not replicate specific author styles without fine-tuning.
Maximum prompt length does not enable style imitation.
Even with 17 poem examples, style transfer remains ineffective.
Abstract
This study examines the ability of GPT-3.5, GPT-3.5-turbo (ChatGPT) and GPT-4 models to generate poems in the style of specific authors using zero-shot and many-shot prompts (which use the maximum context length of 8192 tokens). We assess the performance of models that are not fine-tuned for generating poetry in the style of specific authors, via automated evaluation. Our findings indicate that without fine-tuning, even when provided with the maximum number of 17 poem examples (8192 tokens) in the prompt, these models do not generate poetry in the desired style.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Explainable Artificial Intelligence (XAI) · Machine Learning in Healthcare
MethodsAttention Is All You Need · Cosine Annealing · Linear Layer · Attention Dropout · Position-Wise Feed-Forward Layer · Dense Connections · Refunds@Expedia|||How do I get a full refund from Expedia? · Multi-Head Attention · Adam · {Dispute@FaQ-s}How to file a dispute with Expedia?
