Hiku: Pull-Based Scheduling for Serverless Computing

TL;DR

This paper introduces pull-based scheduling for serverless computing, which improves load balancing, reduces cold starts, and enhances performance by allowing idle workers to request tasks proactively, especially under high concurrency.

Contribution

It proposes a novel pull-based scheduling algorithm that decouples worker selection from task assignment, improving serverless workload performance and load distribution.

Findings

Response latencies improved by 14.9%

Cold start frequency reduced from 43% to 30%

Throughput increased by 8.3%

Abstract

Serverless computing promises convenient abstractions for developing and deploying functions that execute in response to events. In such Function-as-a-Service (FaaS) platforms, scheduling is an integral task, but current scheduling algorithms often struggle with maintaining balanced loads, minimizing cold starts, and adapting to commonly occurring bursty workloads. In this work, we propose pull-based scheduling as a novel scheduling algorithm for serverless computing. Our key idea is to decouple worker selection from task assignment, with idle workers requesting new tasks proactively. Experimental evaluation on an open-source FaaS platform shows that pull-based scheduling, compared to other existing scheduling algorithms, significantly improves the performance and load balancing of serverless workloads, especially under high concurrency. The proposed algorithm improves response latencies by 14.9% compared to hash-based scheduling, reduces the frequency of cold starts from 43% to 30%, increases throughput by 8.3%, and achieves a more even load distribution by 12.9% measured by the requests assigned per worker.