Manga Generation via Layout-controllable Diffusion

TL;DR

This paper introduces a novel method for generating multi-panel manga from plain text, addressing challenges like layout, storytelling coherence, and character consistency, through a new dataset and diffusion-based model.

Contribution

It presents the Manga109Story dataset and MangaDiffusion model, enabling controlled manga generation with diverse layouts and coherent storytelling from textual descriptions.

Findings

Ensures the correct number of panels in generated manga

Produces reasonable and diverse page layouts

Maintains character and semantic consistency

Abstract

Generating comics through text is widely studied. However, there are few studies on generating multi-panel Manga (Japanese comics) solely based on plain text. Japanese manga contains multiple panels on a single page, with characteristics such as coherence in storytelling, reasonable and diverse page layouts, consistency in characters, and semantic correspondence between panel drawings and panel scripts. Therefore, generating manga poses a significant challenge. This paper presents the manga generation task and constructs the Manga109Story dataset for studying manga generation solely from plain text. Additionally, we propose MangaDiffusion to facilitate the intra-panel and inter-panel information interaction during the manga generation process. The results show that our method particularly ensures the number of panels, reasonable and diverse page layouts. Based on our approach, there is potential to converting a large amount of textual stories into more engaging manga readings, leading to significant application prospects.