Managing workflow executions with WESkit

TL;DR

WESkit simplifies managing complex computational workflows in biomedical research by providing a scalable and reproducible solution.

Contribution

WESkit implements the GA4GH WES interface to streamline workflow execution and monitoring across diverse projects.

Findings

WESkit supports Snakemake and Nextflow for consistent automation and centralized monitoring.

The system integrates seamlessly into cloud environments, enhancing collaboration and reproducibility.

WESkit is available under an MIT license with repositories on GitLab and Software Heritage.

Abstract

In biomedical research, managing computational workflows across numerous projects—with varying parameters, tools, and environments—creates major challenges in scalability, reproducibility, and collaboration. Here, we present WESkit, an implementation of the Global Alliance for Genomics and Health (GA4GH) Workflow Execution Service (WES) interface, designed to streamline the execution, monitoring, and documentation of data processing workflows. It addresses the complexities involved in managing numerous executions with varying parameters across diverse research projects. Supporting both Snakemake and Nextflow, the system enables consistent automation and centralized monitoring, which benefits research groups aiming for long-term reproducibility and scalable collaboration. Its suitability for larger teams and service units is further enhanced by seamless integration into cloud environments, contributing to the GA4GH cloud framework.

The software WESkit is available under MIT license at the GitLab repository (https://gitlab.com/one-touch-pipeline/weskit). The WESkit main repository is archived at Software Heritage (https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://gitlab.com/one-touch-pipeline/weskit/api.git) and can be found using “one-touch-pipeline/weskit” term in the search section.