Modeling Story Expectations to Understand Engagement: A Generative Framework Using LLMs

TL;DR

This paper presents a novel framework using large language models to model audience expectations and beliefs about story developments, enhancing understanding of engagement behaviors in narrative media.

Contribution

It introduces a new generative approach leveraging LLMs to capture audience forward-looking beliefs, complementing existing content feature extraction methods.

Findings

Amplifies explanatory power of existing features by 31%

Different engagement types are driven by distinct content expectations

Framework provides insights into how beliefs influence engagement

Abstract

Understanding when and why consumers engage with stories is crucial for content creators and platforms. While existing theories suggest that audience beliefs of what is going to happen should play an important role in engagement decisions, empirical work has mostly focused on developing techniques to directly extract features from actual content, rather than capturing forward-looking beliefs, due to the lack of a principled way to model such beliefs in unstructured narrative data. To complement existing feature extraction techniques, this paper introduces a novel framework that leverages large language models to model audience forward-looking beliefs about how stories might unfold. Our method generates multiple potential continuations for each story and extracts features related to expectations, uncertainty, and surprise using established content analysis techniques. Applying our method to over 30,000 book chapters, we demonstrate that our framework complements existing feature engineering techniques by amplifying their marginal explanatory power on average by 31%. The results reveal that different types of engagement-continuing to read, commenting, and voting-are driven by distinct combinations of current and anticipated content features. Our framework provides a novel way to study and explore how audience forward-looking beliefs shape their engagement with narrative media, with implications for marketing strategy in content-focused industries.