Chiminey: Reliable Computing and Data Management Platform in the Cloud

TL;DR

Chiminey is a versatile cloud platform that simplifies running, managing, and sharing complex scientific computations across different computing environments, addressing failure handling and data visualization needs.

Contribution

It introduces a comprehensive platform that integrates execution management, failure resilience, data curation, visualization, and sharing for scientific computing in the cloud.

Findings

Supports both HPC and cloud infrastructures

Handles execution failures effectively

Facilitates data sharing and visualization

Abstract

The enabling of scientific experiments that are embarrassingly parallel, long running and data-intensive into a cloud-based execution environment is a desirable, though complex undertaking for many researchers. The management of such virtual environments is cumbersome and not necessarily within the core skill set for scientists and engineers. We present here Chiminey, a software platform that enables researchers to (i) run applications on both traditional high-performance computing and cloud-based computing infrastructures, (ii) handle failure during execution, (iii) curate and visualise execution outputs, (iv) share such data with collaborators or the public, and (v) search for publicly available data.