A new job migration algorithm to improve data center efficiency

TL;DR

This paper introduces a novel job migration algorithm designed to optimize resource utilization in heterogeneous data center environments by dynamically rearranging jobs at runtime.

Contribution

It proposes a new method for job migration that enhances resource exploitation in data centers with heterogeneous batch queue systems.

Findings

Increased resource utilization efficiency.

Reduced job waiting times.

Improved overall data center throughput.

Abstract

The under exploitation of the available resources risks to be one of the main problems for a computing center. The growing demand of computational power necessarily entails more complex approaches in the management of the computing resources, with particular attention to the batch queue system scheduler. In a heterogeneous batch queue system, available for both serial single core processes and parallel multi core jobs, it may happen that one or more computational nodes composing the cluster are not fully occupied, running a number of jobs lower than their actual capability. A typical case is represented by more single core jobs running each one over a different multi core server, while more parallel jobs - requiring all the available cores of a host - are queued. A job rearrangement executed at runtime is able to free extra resources, in order to host new processes. We present an efficient method to improve the computing resources exploitation.