# CloudCoaster: Transient-aware Bursty Datacenter Workload Scheduling

## TL;DR

CloudCoaster is a dynamic, transient-aware scheduler for datacenter workloads that adapts resource provisioning in real-time to handle bursty job arrivals, improving performance and reducing costs.

## Contribution

It introduces a novel hybrid scheduler that dynamically resizes clusters using transient servers to better manage bursty workloads.

## Key findings

- Reduces short job queueing delay by 4.8X
- Cuts short partition budget by over 29.5%
- Maintains long job performance during bursts

## Abstract

Today's clusters often have to divide resources among a diverse set of jobs. These jobs are heterogeneous both in execution time and in their rate of arrival. Execution time heterogeneity has lead to the development of hybrid schedulers that can schedule both short and long jobs to ensure good task placement. However, arrival rate heterogeneity, or burstiness, remains a problem in existing schedulers. These hybrid schedulers manage resources on statically provisioned cluster, which can quickly be overwhelmed by bursts in the number of arriving jobs. In this paper we propose CloudCoaster, a hybrid scheduler that dynamically resizes the cluster by leveraging cheap transient servers. CloudCoaster schedules jobs in an intelligent way that increases job performance while reducing overall resource cost. We evaluate the effectiveness of CloudCoaster through simulations on real-world traces and compare it against a state-of-art hybrid scheduler. CloudCoaster improves the average queueing delay time of short jobs by 4.8X while maintaining long job performance. In addition, CloudCoaster reduces the short partition budget by over 29.5%.