StoryScope: Investigating idiosyncrasies in AI fiction
Jenna Russell, Rishanth Rajendhran, Chau Minh Pham, Mohit Iyyer, John Wieting

TL;DR
This paper introduces StoryScope, a method to distinguish AI-generated fiction from human stories based on discourse-level narrative features, achieving high accuracy without relying on stylistic cues.
Contribution
StoryScope is a novel pipeline that automatically extracts interpretable narrative features, enabling effective AI vs. human story classification and authorship attribution.
Findings
Narrative features alone achieve 93.2% macro-F1 in human vs. AI detection.
A core set of 30 features captures key differences in storytelling.
AI stories tend to over-explain themes and have simpler plots.
Abstract
As AI-generated fiction becomes increasingly prevalent, questions of authorship and originality are becoming central to how written work is evaluated. While most existing work in this space focuses on identifying surface-level signatures of AI writing, we ask instead whether AI-generated stories can be distinguished from human ones without relying on stylistic signals, focusing on discourse-level narrative choices such as character agency and chronological discontinuity. We propose StoryScope, a pipeline that automatically induces a fine-grained, interpretable feature space of discourse-level narrative features across 10 dimensions. We apply StoryScope to a parallel corpus of 10,272 writing prompts, each written by a human author and five LLMs, yielding 61,608 stories, each ~5,000 words, and 304 extracted features per story. Narrative features alone achieve 93.2% macro-F1 for human vs.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
