Exploiting Spot Instances for Time-Critical Cloud Workloads Using Optimal Randomized Strategies

TL;DR

This paper introduces ROSS, a randomized scheduling algorithm that optimally balances cost and deadlines for cloud jobs using spot and on-demand instances, outperforming existing methods.

Contribution

The paper proves a fundamental limit for deterministic policies and proposes ROSS, a randomized algorithm with optimal competitive ratio for deadline-aware cloud scheduling.

Findings

ROSS achieves a competitive ratio of √K, improving over deterministic policies.

Extensive evaluations show ROSS saves up to 30% costs compared to state-of-the-art.

ROSS effectively balances cost and deadline guarantees in real-world cloud environments.

Abstract

This paper addresses the challenge of deadline-aware online scheduling for jobs in hybrid cloud environments, where jobs may run on either cost-effective but unreliable spot instances or more expensive on-demand instances, under hard deadlines. We first establish a fundamental limit for existing (predominantly-) deterministic policies, proving a worst-case competitive ratio of $\Omega(K)$, where $K$ is the cost ratio between on-demand and spot instances. We then present a novel randomized scheduling algorithm, ROSS, that achieves a provably optimal competitive ratio of $\sqrt{K}$ under reasonable deadlines, significantly improving upon existing approaches. Extensive evaluations on real-world trace data from Azure and AWS demonstrate that ROSS effectively balances cost optimization and deadline guarantees, consistently outperforming the state-of-the-art by up to $30\%$ in cost savings, across diverse spot market conditions.