Visualizing the Scripts of Data Wrangling with SOMNUS

TL;DR

This paper introduces Somnus, a visualization tool that helps users understand data transformation scripts by illustrating their semantics and evolution, improving accuracy and efficiency over textual descriptions.

Contribution

We propose a novel visualization design space and a pipeline, Somnus, for interactive exploration of data transformation scripts and their effects, with positive user feedback.

Findings

Participants achieved better accuracy with less time using Somnus.

Users preferred Somnus over textual descriptions.

Somnus effectively visualizes data transformation semantics.

Abstract

Data workers use various scripting languages for data transformation, such as SAS, R, and Python. However, understanding intricate code pieces requires advanced programming skills, which hinders data workers from grasping the idea of data transformation at ease. Program visualization is beneficial for debugging and education and has the potential to illustrate transformations intuitively and interactively. In this paper, we explore visualization design for demonstrating the semantics of code pieces in the context of data transformation. First, to depict individual data transformations, we structure a design space by two primary dimensions, i.e., key parameters to encode and possible visual channels to be mapped. Then, we derive a collection of 23 glyphs that visualize the semantics of transformations. Next, we design a pipeline, named Somnus, that provides an overview of the creation and evolution of data tables using a provenance graph. At the same time, it allows detailed investigation of individual transformations. User feedback on Somnus is positive. Our study participants achieved better accuracy with less time using Somnus, and preferred it over carefully-crafted textual description. Further, we provide two example applications to demonstrate the utility and versatility of Somnus.