On-Demand Virtual Research Environments using Microservices

TL;DR

This paper introduces a flexible, vendor-agnostic microservice-based approach for creating on-demand virtual research environments using containers, enhancing scalability and management of scientific workflows across cloud platforms.

Contribution

It presents a novel microservice-oriented methodology and open source implementation for scalable, on-demand scientific environments compatible with major cloud providers.

Findings

Demonstrated scalability in life science applications

Vendor-agnostic platform supporting major cloud providers

Effective management of scientific pipelines in distributed environments

Abstract

The computational demands for scientific applications are continuously increasing. The emergence of cloud computing has enabled on-demand resource allocation. However, relying solely on infrastructure as a service does not achieve the degree of flexibility required by the scientific community. Here we present a microservice-oriented methodology, where scientific applications run in a distributed orchestration platform as software containers, referred to as on-demand, virtual research environments. The methodology is vendor agnostic and we provide an open source implementation that supports the major cloud providers, offering scalable management of scientific pipelines. We demonstrate applicability and scalability of our methodology in life science applications, but the methodology is general and can be applied to other scientific domains.