Distributed Join-the-Idle-Queue for Low Latency Cloud Services

TL;DR

This paper analyzes and improves distributed Join-the-Idle-Queue (JIQ) algorithms to achieve low latency in large-scale cloud services, providing delay performance expressions and proposing a superior variant.

Contribution

It offers a detailed delay performance analysis of distributed JIQ and introduces a new variant with advantages for large-scale systems.

Findings

Derived semi-closed form delay expressions for distributed JIQ

Proposed a new JIQ variant with improved performance

Demonstrated advantages over existing algorithms for large systems

Abstract

Low latency is highly desirable for cloud services. To achieve low response time, stringent timing requirements are needed for task scheduling in a large-scale server farm spanning thousands of servers. In this paper, we conduct an in-depth analysis for distributed Join-the-Idle-Queue (JIQ), a promising new approximation of an idealized task-scheduling algorithm. In particular, we derive semi-closed form expressions for the delay performance of distributed JIQ, and we propose a new variant of distributed JIQ that offers clear advantages over alternative algorithms for large systems.