The Technologies Required for Fusing HPC and Real-Time Data to Support Urgent Computing

TL;DR

This paper discusses the technological challenges and solutions for integrating HPC with real-time data to enable urgent computing for disaster response, focusing on the VESTEC project's prototype development.

Contribution

It presents the design and analysis of the VESTEC urgent computing control system, addressing obstacles like queue wait times and technology integration.

Findings

Analysis of available technologies for urgent HPC computing

Design of a prototype urgent computing control system

Identification of ongoing challenges for production deployment

Abstract

The use of High Performance Computing (HPC) to compliment urgent decision making in the event of disasters is an important future potential use of supercomputers. However, the usage modes involved are rather different from how HPC has been used traditionally. As such, there are many obstacles that need to be overcome, not least the unbounded wait times in the batch system queues, to make the use of HPC in disaster response practical. In this paper, we present how the VESTEC project plans to overcome these issues and develop a working prototype of an urgent computing control system. We describe the requirements for such a system and analyse the different technologies available that can be leveraged to successfully build such a system. We finally explore the design of the VESTEC system and discuss ongoing challenges that need to be addressed to realise a production level system.