A Workflow Management System Guide
Caspar Schmitt, Boyang Yu, Thomas Kuhr

TL;DR
This paper reviews and compares various workflow management tools used in scientific analysis to aid researchers in selecting appropriate solutions for transparency, reproducibility, and data preservation.
Contribution
It provides a comprehensive comparison of workflow management tools and offers recommendations tailored to different scientific use cases.
Findings
Identifies key criteria for workflow management tools
Provides a comparative analysis of selected tools
Recommends suitable tools for specific research needs
Abstract
A workflow describes the entirety of processing steps in an analysis, such as employed in many fields of physics. Workflow management makes the dependencies between individual steps of a workflow and their computational requirements explicit, such that entire workflows can be executed in a stand-alone manner. Though the use of workflow management is widely recommended in the interest of transparency, reproducibility and data preservation, choosing among the large variety of available workflow management tools can be overwhelming. We compare selected workflow management tools concerning all relevant criteria and make recommendations for different use cases.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsScientific Computing and Data Management · Distributed and Parallel Computing Systems · Advanced Data Storage Technologies
