DeckFlow: Iterative Specification on a Multimodal Generative Canvas

TL;DR

DeckFlow is a multimodal generative AI tool that enables iterative task and specification decomposition on an infinite canvas, supporting creative workflows with text, image, and audio generation, and allowing recursive feedback for design refinement.

Contribution

It introduces DeckFlow, a novel multimodal AI platform that addresses key limitations in existing tools by supporting task decomposition, specification refinement, and generative space exploration.

Findings

DeckFlow outperforms a conversational AI baseline in text-to-image tasks.

Users effectively utilize DeckFlow for open-ended creative projects.

DeckFlow facilitates iterative design with multimodal outputs.

Abstract

Generative AI promises to allow people to create high-quality personalized media. Although powerful, we identify three fundamental design problems with existing tooling through a literature review. We introduce a multimodal generative AI tool, DeckFlow, to address these problems. First, DeckFlow supports task decomposition by allowing users to maintain multiple interconnected subtasks on an infinite canvas populated by cards connected through visual dataflow affordances. Second, DeckFlow supports a specification decomposition workflow where an initial goal is iteratively decomposed into smaller parts and combined using feature labels and clusters. Finally, DeckFlow supports generative space exploration by generating multiple prompt and output variations, presented in a grid, that can feed back recursively into the next design iteration. We evaluate DeckFlow for text-to-image generation against a state-of-practice conversational AI baseline for image generation tasks. We then add audio generation and investigate user behaviors in a more open-ended creative setting with text, image, and audio outputs.