LiteraryTaste: A Preference Dataset for Creative Writing Personalization
John Joon Young Chung, Vishakh Padmakumar, Melissa Roemmele, Yi Wang, Yuqian Sun, Tiffany Wang, Shm Garanganao Almeda, Brett A. Halperin, Yuwen Lu, Max Kreminski

TL;DR
LiteraryTaste introduces a dataset capturing individual reading preferences to enhance personalization in creative writing models, revealing diverse tastes and limited correlation between stated and actual preferences.
Contribution
The paper presents LiteraryTaste, a novel dataset with self-reported and annotated preferences, and demonstrates how fine-tuned transformers can model personal creative writing tastes.
Findings
People have diverse creative writing preferences.
Fine-tuned transformers achieve over 75% accuracy in modeling preferences.
Stated preferences are less predictive of actual preferences.
Abstract
People have different creative writing preferences, and large language models (LLMs) for these tasks can benefit from adapting to each user's preferences. However, these models are often trained over a dataset that considers varying personal tastes as a monolith. To facilitate developing personalized creative writing LLMs, we introduce LiteraryTaste, a dataset of reading preferences from 60 people, where each person: 1) self-reported their reading habits and tastes (stated preference), and 2) annotated their preferences over 100 pairs of short creative writing texts (revealed preference). With our dataset, we found that: 1) people diverge on creative writing preferences, 2) finetuning a transformer encoder could achieve 75.8% and 67.7% accuracy when modeling personal and collective revealed preferences, and 3) stated preferences had limited utility in modeling revealed preferences. With…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Games · Recommender Systems and Techniques · Aesthetic Perception and Analysis
