Workflows to driving high-performance interactive supercomputing for urgent decision making

TL;DR

This paper investigates the use of workflows to enhance the performance and flexibility of interactive supercomputing for urgent decision-making, focusing on connecting users, codes, and data sources effectively.

Contribution

It introduces a dual-workflow system approach for urgent workloads, demonstrating how interoperability improves supercomputing responsiveness and utility in critical scenarios.

Findings

Interoperable workflows improve urgent computing efficiency.

Dual workflow systems enable better resource management.

Case study on space weather prediction illustrates benefits.

Abstract

Interactive urgent computing is a small but growing user of supercomputing resources. However there are numerous technical challenges that must be overcome to make supercomputers fully suited to the wide range of urgent workloads which could benefit from the computational power delivered by such instruments. An important question is how to connect the different components of an urgent workload; namely the users, the simulation codes, and external data sources, together in a structured and accessible manner. In this paper we explore the role of workflows from both the perspective of marshalling and control of urgent workloads, and at the individual HPC machine level. Ultimately requiring two workflow systems, by using a space weather prediction urgent use-cases, we explore the benefit that these two workflow systems provide especially when one exploits the flexibility enabled by them interoperating.