Enabling Adaptive Grid Scheduling and Resource Management

TL;DR

This paper introduces measurement tools and future plans for a probabilistic Grid scheduler to improve resource management and workflow scheduling in heterogeneous, federated computing environments.

Contribution

It presents new measurement applications for resource monitoring and outlines a customizable emulator, advancing adaptive scheduling in Grid systems.

Findings

Developed per-process resource utilisation monitoring tools

Created a customizable resource utilisation emulator

Outlined future work on a predictive, probabilistic scheduler

Abstract

Wider adoption of the Grid concept has led to an increasing amount of federated computational, storage and visualisation resources being available to scientists and researchers. Distributed and heterogeneous nature of these resources renders most of the legacy cluster monitoring and management approaches inappropriate, and poses new challenges in workflow scheduling on such systems. Effective resource utilisation monitoring and highly granular yet adaptive measurements are prerequisites for a more efficient Grid scheduler. We present a suite of measurement applications able to monitor per-process resource utilisation, and a customisable tool for emulating observed utilisation models. We also outline our future work on a predictive and probabilistic Grid scheduler. The research is undertaken as part of UK e-Science EPSRC sponsored project SO-GRM (Self-Organising Grid Resource Management) in cooperation with BT.