Characterizing the Quality of Insight by Interactions: A Case Study

TL;DR

This case study investigates how user interactions with a visualization tool can be used to assess the quality of insights generated during data exploration, highlighting the potential for interaction-based insight qualification.

Contribution

The paper introduces a method to characterize insight quality through interaction patterns in a visualization tool, supported by empirical data from user exploration sessions.

Findings

Exploration actions can lead to unexpected insights.

Drill-down patterns tend to increase domain value of insights.

Using domain knowledge guides exploration and improves insight quality.

Abstract

Understanding the quality of insight has become increasingly important with the trend of allowing users to post comments during visual exploration, yet approaches for qualifying insight are rare. This paper presents a case study to investigate the possibility of characterizing the quality of insight via the interactions performed. To do this, we devised the interaction of a visualization tool-MediSyn-for insight generation. MediSyn supports five types of interactions: selecting, connecting, elaborating, exploring, and sharing. We evaluated MediSyn with 14 participants by allowing them to freely explore the data and generate insights. We then extracted seven interaction patterns from their interaction logs and correlated the patterns to four aspects of insight quality. The results show the possibility of qualifying insights via interactions. Among other findings, exploration actions can lead to unexpected insights; the drill-down pattern tends to increase the domain values of insights. A qualitative analysis shows that using domain knowledge to guide exploration can positively affect the domain value of derived insights. We discuss the study's implications, lessons learned, and future research opportunities.