A Case Study: Using Genetic Algorithm for Job Scheduling Problem

TL;DR

This paper applies a genetic algorithm to optimize job scheduling in complex DevOps pipelines, aiming to reduce makespan and resource usage by automatically prioritizing jobs and allocating resources.

Contribution

It introduces a multi-objective genetic algorithm approach for automatic job prioritization and resource allocation in job scheduling, improving efficiency over existing methods.

Findings

The proposed GA-based method reduces makespan more effectively.

It decreases the number of machines required for job completion.

The approach outperforms current priority list methods in experiments.

Abstract

Nowadays, DevOps pipelines of huge projects are getting more and more complex. Each job in the pipeline might need different requirements including specific hardware specifications and dependencies. To achieve minimal makespan, developers always apply as much machines as possible. Consequently, others may be stalled for waiting resource released. Minimizing the makespan of each job using a few resource is a challenging problem. In this study, it is aimed to 1) automatically determine the priority of jobs to reduce the waiting time in the line, 2) automatically allocate the machine resource to each job. In this work, the problem is formulated as a multi-objective optimization problem. We use GA algorithm to automatically determine job priorities and resource demand for minimizing individual makespan and resource usage. Finally, the experimental results show that our proposed priority list generation algorithm is more effective than current priority list producing method in the aspects of makespan and allocated machine count.