Data Playwright: Authoring Data Videos with Annotated Narration

TL;DR

Data Playwright introduces a novel system that enables users to create data videos by seamlessly integrating narration and design commands through annotated narration, leveraging large language models for synthesis.

Contribution

The paper presents a new paradigm and prototype system that unify narrative content and design commands in data video creation, addressing previous limitations.

Findings

Users can effectively create data videos with minimal effort.

Annotated narration improves customization and narrative coherence.

Large language models enable accurate synthesis of videos from annotated inputs.

Abstract

Creating data videos that effectively narrate stories with animated visuals requires substantial effort and expertise. A promising research trend is leveraging the easy-to-use natural language (NL) interaction to automatically synthesize data video components from narrative content like text narrations, or NL commands that specify user-required designs. Nevertheless, previous research has overlooked the integration of narrative content and specific design authoring commands, leading to generated results that lack customization or fail to seamlessly fit into the narrative context. To address these issues, we introduce a novel paradigm for creating data videos, which seamlessly integrates users' authoring and narrative intents in a unified format called annotated narration, allowing users to incorporate NL commands for design authoring as inline annotations within the narration text. Informed by a formative study on users' preference for annotated narration, we develop a prototype system named Data Playwright that embodies this paradigm for effective creation of data videos. Within Data Playwright, users can write annotated narration based on uploaded visualizations. The system's interpreter automatically understands users' inputs and synthesizes data videos with narration-animation interplay, powered by large language models. Finally, users can preview and fine-tune the video. A user study demonstrated that participants can effectively create data videos with Data Playwright by effortlessly articulating their desired outcomes through annotated narration.