Comprehending Spatio-temporal Data via Cinematic Storytelling using Large Language Models

TL;DR

This paper introduces MapMuse, a novel storytelling framework that uses cinematic principles and large language models to interpret and communicate complex spatio-temporal data effectively, enhancing understanding and engagement.

Contribution

It presents a new narrative-driven approach leveraging LLMs and cinematic storytelling techniques to interpret and visualize spatio-temporal datasets for broader audiences.

Findings

MapMuse effectively visualizes urban mobility patterns.

Narrative techniques improve data engagement and understanding.

Case study demonstrates potential for broader application.

Abstract

Spatio-temporal data captures complex dynamics across both space and time, yet traditional visualizations are complex, require domain expertise and often fail to resonate with broader audiences. Here, we propose MapMuse, a storytelling-based framework for interpreting spatio-temporal datasets, transforming them into compelling, narrative-driven experiences. We utilize large language models and employ retrieval augmented generation (RAG) and agent-based techniques to generate comprehensive stories. Drawing on principles common in cinematic storytelling, we emphasize clarity, emotional connection, and audience-centric design. As a case study, we analyze a dataset of taxi trajectories. Two perspectives are presented: a captivating story based on a heat map that visualizes millions of taxi trip endpoints to uncover urban mobility patterns; and a detailed narrative following a single long taxi journey, enriched with city landmarks and temporal shifts. By portraying locations as characters and movement as plot, we argue that data storytelling drives insight, engagement, and action from spatio-temporal information. The case study illustrates how MapMuse can bridge the gap between data complexity and human understanding. The aim of this short paper is to provide a glimpse to the potential of the cinematic storytelling technique as an effective communication tool for spatio-temporal data, as well as to describe open problems and opportunities for future research.