StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration
Panwen Hu, Jin Jiang, Jianqi Chen, Mingfei Han, Shengcai Liao, Xiaojun, Chang, Xiaodan Liang

TL;DR
StoryAgent is a multi-agent framework that improves customized storytelling video generation by ensuring subject consistency and integrating specialized agents for story design, storyboard creation, and video synthesis.
Contribution
It introduces a novel multi-agent system for CSVG, enhancing control and consistency, and proposes new methods like LoRA-BE for intra-shot temporal coherence.
Findings
Outperforms state-of-the-art methods in consistency metrics
Effectively maintains protagonist identity across shots
Enhances control over storytelling video synthesis
Abstract
The advent of AI-Generated Content (AIGC) has spurred research into automated video generation to streamline conventional processes. However, automating storytelling video production, particularly for customized narratives, remains challenging due to the complexity of maintaining subject consistency across shots. While existing approaches like Mora and AesopAgent integrate multiple agents for Story-to-Video (S2V) generation, they fall short in preserving protagonist consistency and supporting Customized Storytelling Video Generation (CSVG). To address these limitations, we propose StoryAgent, a multi-agent framework designed for CSVG. StoryAgent decomposes CSVG into distinct subtasks assigned to specialized agents, mirroring the professional production process. Notably, our framework includes agents for story design, storyboard generation, video creation, agent coordination, and result…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Motion and Animation · Video Analysis and Summarization · Artificial Intelligence in Games
