Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization

TL;DR

Patchview is an interactive visualization system that leverages large language models to assist writers in worldbuilding by visually representing and steering generated story elements through a metaphor of magnets and dust.

Contribution

We introduce Patchview, a novel visual interface that enhances sensemaking and control of LLM-generated world elements in storytelling.

Findings

Supports effective sensemaking of world elements

Enables user steering of generated content

Aligns LLM behaviors with user intentions

Abstract

Large language models (LLMs) can help writers build story worlds by generating world elements, such as factions, characters, and locations. However, making sense of many generated elements can be overwhelming. Moreover, if the user wants to precisely control aspects of generated elements that are difficult to specify verbally, prompting alone may be insufficient. We introduce Patchview, a customizable LLM-powered system that visually aids worldbuilding by allowing users to interact with story concepts and elements through the physical metaphor of magnets and dust. Elements in Patchview are visually dragged closer to concepts with high relevance, facilitating sensemaking. The user can also steer the generation with verbally elusive concepts by indicating the desired position of the element between concepts. When the user disagrees with the LLM's visualization and generation, they can correct those by repositioning the element. These corrections can be used to align the LLM's future behaviors to the user's perception. With a user study, we show that Patchview supports the sensemaking of world elements and steering of element generation, facilitating exploration during the worldbuilding process. Patchview provides insights on how customizable visual representation can help sensemake, steer, and align generative AI model behaviors with the user's intentions.