or2yw: Modeling and Visualizing OpenRefineHistories as YesWorkflow Diagrams

TL;DR

This paper introduces or2yw, a tool that converts OpenRefine data cleaning histories into YesWorkflow diagrams, enabling visualization, querying, and better understanding of workflows for transparency and reuse.

Contribution

The paper presents a novel method to automatically generate YesWorkflow models from OpenRefine operation histories, including linear and parallel workflow visualizations.

Findings

Generated models improve workflow transparency.

Parallel models reveal independent data cleaning steps.

Tool facilitates workflow reuse and documentation.

Abstract

OpenRefine is a popular open-source data cleaning tool. It allows users to export a previously executed data cleaning workflow in a JSON format for possible reuse on other datasets. We have developed or2yw, a novel tool that maps a JSON-formatted OpenRefine operation history to a YesWorkflow (YW) model, which then can be visualized and queried using the YW tool. The latter was originally developed to allow researchers a simple way to annotate their program scripts in order to reveal the workflow steps and dataflow dependencies implicit in those scripts. With or2yw the user can automatically generate YW models from OpenRefine operation histories, thus providing a 'workflow view' on a previously executed sequence of data cleaning operations. The or2yw tool can generate different types of YesWorkflow models, e.g., a linear model which mirrors the sequential execution order of operations in OpenRefine, and a \emph{parallel model} which reveals independent workflow branches, based on a simple analysis of dependencies between steps: if two operations are independent of each other (e.g., when the columns they read and write do not overlap) then these can be viewed as parallel steps in the data cleaning workflow. The resulting YW models can be understood as a form of prospective provenance, i.e., knowledge artifacts that can be queried and visualized (i) to help authors document their own data cleaning workflows, thereby increasing transparency, and (ii) to help other users, who might want to reuse such workflows, to understand them better.