InSituTale: Enhancing Augmented Data Storytelling with Physical Objects
Kentaro Takahira, Yue Yu, Takanori Fujiwara, Ryo Suzuki, Huamin Qu

TL;DR
InSituTale introduces a novel augmented data storytelling system that allows presenters to manipulate visualizations through physical objects, enhancing engagement and interaction by combining physical manipulations with digital visualization control.
Contribution
The paper presents InSituTale, a new system that integrates physical object interactions with visualization control, informed by surveys and workshops with HCI/VIS researchers.
Findings
Participants found InSituTale intuitive and engaging.
The system effectively combines physical and digital interactions.
InSituTale enhances the storytelling experience.
Abstract
Augmented data storytelling enhances narrative delivery by integrating visualizations with physical environments and presenter actions. Existing systems predominantly rely on body gestures or speech to control visualizations, leaving interactions with physical objects largely underexplored. We introduce augmented physical data storytelling, an approach enabling presenters to manipulate visualizations through physical object interactions. To inform this approach, we first conducted a survey of data-driven presentations to identify common visualization commands. We then conducted workshops with nine HCI/VIS researchers to collect mappings between physical manipulations and these commands. Guided by these insights, we developed InSituTale, a prototype that combines object tracking via a depth camera with Vision-LLM for detecting real-world events. Through physical manipulations, presenters…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
