Enabling and Optimizing Pilot Jobs using Xen based Virtual Machines for the HPC Grid Applications

TL;DR

This paper proposes a virtualization approach integrated with Pilot Jobs to improve resource management and customization in HPC Grid applications, requiring minimal changes from resource providers.

Contribution

It introduces a novel method combining Xen-based virtualization with Pilot Jobs for efficient resource utilization in HPC Grids, with minimal infrastructure modifications.

Findings

Enhanced resource isolation and management

Improved environment customization for users

Minimal changes needed for existing infrastructure

Abstract

The primary motivation for uptake of virtualization have been resource isolation, capacity management and resource customization: isolation and capacity management allow providers to isolate users from the site and control their resources usage while customization allows end-users to easily project the required environment onto a variety of sites. Various approaches have been taken to integrate virtualization with Grid technologies. In this paper, we propose an approach that combines virtualization on the existing software infrastructure such as Pilot Jobs with minimum change on the part of resource providers.