Designing for Recommending Intermediate States in A Scientific Workflow Management System

TL;DR

This paper introduces GUI-RISPTS, a novel GUI-based approach integrated into SciWorCS to recommend intermediate data states, aiming to improve workflow efficiency and user experience in scientific workflow management systems.

Contribution

It presents a new GUI-based technique for managing and recommending intermediate data states in SWfMS, with integration and evaluation in SciWorCS.

Findings

GUI-RISPTS reduces computational time for certain modules.

User evaluation shows improved workflow management experience.

Overhead in storage and efficiency is acceptable.

Abstract

To process a large amount of data sequentially and systematically, proper management of workflow components (i.e., modules, data, configurations, associations among ports and links) in a Scientific Workflow Management System (SWfMS) is inevitable. Managing data with provenance in a SWfMS to support reusability of workflows, modules, and data is not a simple task. Handling such components is even more burdensome for frequently assembled and executed complex workflows for investigating large datasets with different technologies (i.e., various learning algorithms or models). However, a great many studies propose various techniques and technologies for managing and recommending services in a SWfMS, but only a very few studies consider the management of data in a SWfMS for efficient storing and facilitating workflow executions. Furthermore, there is no study to inquire about the effectiveness and efficiency of such data management in a SWfMS from a user perspective. In this paper, we present and evaluate a GUI version of such a novel approach of intermediate data management with two use cases (Plant Phenotyping and Bioinformatics). The technique we call GUI-RISPTS (Recommending Intermediate States from Pipelines Considering Tool-States) can facilitate executions of workflows with processed data (i.e., intermediate outcomes of modules in a workflow) and can thus reduce the computational time of some modules in a SWfMS. We integrated GUI-RISPTS with an existing workflow management system called SciWorCS. In SciWorCS, we present an interface that users use for selecting the recommendation of intermediate states (i.e., modules' outcomes). We investigated GUI-RISP's effectiveness from users' perspectives along with measuring its overhead in terms of storage and efficiency in workflow execution.