The Influence of Visual Provenance Representations on Strategies in a Collaborative Hand-off Data Analysis Scenario

TL;DR

This study examines how different types of visual provenance information, such as data coverage and interaction history, influence analysts' confidence, strategies, and behaviors during collaborative data analysis tasks.

Contribution

It provides empirical insights into how specific provenance representations affect analyst confidence and investigation strategies in open-ended scenarios.

Findings

Data coverage increases confidence without restricting exploration.

Interaction history requires more time to interpret and reduces confidence.

Provenance presentation influences analysis behaviors and decision-making.

Abstract

Conducting data analysis tasks rarely occur in isolation. Especially in intelligence analysis scenarios where different experts contribute knowledge to a shared understanding, members must communicate how insights develop to establish common ground among collaborators. The use of provenance to communicate analytic sensemaking carries promise by describing the interactions and summarizing the steps taken to reach insights. Yet, no universal guidelines exist for communicating provenance in different settings. Our work focuses on the presentation of provenance information and the resulting conclusions reached and strategies used by new analysts. In an open-ended, 30-minute, textual exploration scenario, we qualitatively compare how adding different types of provenance information (specifically data coverage and interaction history) affects analysts' confidence in conclusions developed, propensity to repeat work, filtering of data, identification of relevant information, and typical investigation strategies. We see that data coverage (i.e., what was interacted with) provides provenance information without limiting individual investigation freedom. On the other hand, while interaction history (i.e., when something was interacted with) does not significantly encourage more mimicry, it does take more time to comfortably understand, as represented by less confident conclusions and less relevant information-gathering behaviors. Our results contribute empirical data towards understanding how provenance summarizations can influence analysis behaviors.