Data Player: Automatic Generation of Data Videos with Narration-Animation Interplay

TL;DR

Data Player automates the creation of data videos with narration and animation interplay, reducing manual effort and technical skill requirements, while producing high-quality, data-driven videos comparable to human-made ones.

Contribution

This paper introduces Data Player, a novel system that automatically generates data videos with synchronized narration and animations using large language models and computational constraints.

Findings

Data Player produces videos comparable to human-created ones.

User studies and expert interviews validate the quality of generated videos.

The system effectively links visual data with narration through semantic and animation constraints.

Abstract

Data visualizations and narratives are often integrated to convey data stories effectively. Among various data storytelling formats, data videos have been garnering increasing attention. These videos provide an intuitive interpretation of data charts while vividly articulating the underlying data insights. However, the production of data videos demands a diverse set of professional skills and considerable manual labor, including understanding narratives, linking visual elements with narration segments, designing and crafting animations, recording audio narrations, and synchronizing audio with visual animations. To simplify this process, our paper introduces a novel method, referred to as Data Player, capable of automatically generating dynamic data videos with narration-animation interplay. This approach lowers the technical barriers associated with creating data videos rich in narration. To enable narration-animation interplay, Data Player constructs references between visualizations and text input. Specifically, it first extracts data into tables from the visualizations. Subsequently, it utilizes large language models to form semantic connections between text and visuals. Finally, Data Player encodes animation design knowledge as computational low-level constraints, allowing for the recommendation of suitable animation presets that align with the audio narration produced by text-to-speech technologies. We assessed Data Player's efficacy through an example gallery, a user study, and expert interviews. The evaluation results demonstrated that Data Player can generate high-quality data videos that are comparable to human-composed ones.