Semantic Interaction for Narrative Map Sensemaking: An Insight-based Evaluation

TL;DR

This study empirically evaluates semantic interaction in narrative map sensemaking, demonstrating its effectiveness over timelines and revealing two distinct SI approaches through a user study with 33 participants.

Contribution

It provides the first empirical evidence that semantic interaction enhances narrative map sensemaking and introduces qualitative insights into SI usage strategies.

Findings

Map-based prototypes outperform timelines in insight generation.

SI-enabled condition shows the highest mean performance, with large effect sizes.

Two SI approaches, corrective and additive, were identified.

Abstract

Semantic interaction (SI) enables analysts to incorporate their cognitive processes into AI models through direct manipulation of visualizations. While SI frameworks for narrative extraction have been proposed, empirical evaluations of their effectiveness remain limited. This paper presents a user study that evaluates SI for narrative map sensemaking, involving 33 participants under three conditions: a timeline baseline, a basic narrative map, and an interactive narrative map with SI capabilities. The results show that the map-based prototypes yielded more insights than the timeline baseline, with the SI-enabled condition reaching statistical significance and the basic map condition trending in the same direction. The SI-enabled condition showed the highest mean performance; differences between the map conditions were not statistically significant but showed large effect sizes (d > 0.8), suggesting that the study was underpowered to detect them. Qualitative analysis identified two distinct SI approaches-corrective and additive-that enable analysts to impose quality judgments and organizational structure on extracted narratives. We also find that SI users achieved comparable exploration breadth with less parameter manipulation, suggesting that SI serves as an alternative pathway for model refinement. This work provides empirical evidence that map-based representations outperform timelines for narrative sensemaking, along with qualitative insights into how analysts use SI for narrative refinement.