EmoStory: Emotion-Aware Story Generation

TL;DR

EmoStory introduces an emotion-aware story generation framework that creates coherent visual narratives with explicit emotional directions, enhancing engagement by grounding emotions in visual elements and maintaining subject consistency.

Contribution

This paper presents EmoStory, the first framework for emotion-aware story generation that integrates story planning and region-aware visual synthesis to incorporate explicit emotions.

Findings

Outperforms existing methods in emotion accuracy.

Achieves higher subject consistency in generated stories.

User studies favor EmoStory's emotional and narrative quality.

Abstract

Story generation aims to produce image sequences that depict coherent narratives while maintaining subject consistency across frames. Although existing methods have excelled in producing coherent and expressive stories, they remain largely emotion-neutral, focusing on what subject appears in a story while overlooking how emotions shape narrative interpretation and visual presentation. As stories are intended to engage audiences emotionally, we introduce emotion-aware story generation, a new task that aims to generate subject-consistent visual stories with explicit emotional directions. This task is challenging due to the abstract nature of emotions, which must be grounded in concrete visual elements and consistently expressed across a narrative through visual composition. To address these challenges, we propose EmoStory, a two-stage framework that integrates agent-based story planning and region-aware story generation. The planning stage transforms target emotions into coherent story prompts with emotion agent and writer agent, while the generation stage preserves subject consistency and injects emotion-related elements through region-aware composition. We evaluate EmoStory on a newly constructed dataset covering 25 subjects and 600 emotional stories. Extensive quantitative and qualitative results, along with user studies, show that EmoStory outperforms state-of-the-art story generation methods in emotion accuracy, prompt alignment, and subject consistency.