Resource management on a VM based computer cluster for scientific computing

TL;DR

This paper discusses the implementation of a virtual host-based resource management system for scientific computing clusters, focusing on storage, middleware, and resource allocation strategies to optimize HPC and HTC workloads.

Contribution

It presents a customized virtualization approach for HPC/HTC resource management, detailing storage, middleware, and allocation strategies at INFN's Gran Sasso Laboratory.

Findings

Virtualization reduces performance barriers in HPC/HTC environments.

Customized resource management strategies improve computational efficiency.

The approach is tailored to specific infrastructure needs.

Abstract

In the last ten years host virtualization has brought a revolution in the way almost every activity related to information technology is thought of and performed. The use of virtualization for HPC and HTC computing, while eagerly desired, has probably been one of the last steps of this revolution, the performance loss due to the hardware abstraction layer being the cause that slowed down a process that has been much faster in other fields. Nowadays the widespread diffusion of virtualization and of new virtualization techniques seem to have helped breaking this last barrier and virtual host computing infrastructures for HPC and HTC are found in many data centers. In this document the approach adopted at the INFN "Laboratori Nazionali del Gran Sasso" for providing computational resources via a virtual host based computing facility is described. Particular evidence is given to the storage layout, to the middleware architecture and to resource allocation strategies, as these are issues for which a personalized solution was adopted. Other aspects may be covered in the future within other documents.