TaleForge: Interactive Multimodal System for Personalized Story Creation

TL;DR

TaleForge is a multimodal storytelling system that personalizes narratives and illustrations by integrating user facial images and preferences, enhancing engagement and immersion in story creation.

Contribution

It introduces a novel system combining LLMs and text-to-image diffusion for personalized, interactive storytelling with real-time previews and user control.

Findings

Increased user engagement and sense of ownership.

Positive feedback on real-time previews and controls.

Finer narrative editing tools requested by users.

Abstract

Storytelling is a deeply personal and creative process, yet existing methods often treat users as passive consumers, offering generic plots with limited personalization. This undermines engagement and immersion, especially where individual style or appearance is crucial. We introduce TaleForge, a personalized story-generation system that integrates large language models (LLMs) and text-to-image diffusion to embed users' facial images within both narratives and illustrations. TaleForge features three interconnected modules: Story Generation, where LLMs create narratives and character descriptions from user prompts; Personalized Image Generation, merging users' faces and outfit choices into character illustrations; and Background Generation, creating scene backdrops that incorporate personalized characters. A user study demonstrated heightened engagement and ownership when individuals appeared as protagonists. Participants praised the system's real-time previews and intuitive controls, though they requested finer narrative editing tools. TaleForge advances multimodal storytelling by aligning personalized text and imagery to create immersive, user-centric experiences.