Tiny Autoscalers for Tiny Workloads: Dynamic CPU Allocation for Serverless Functions

TL;DR

This paper explores lightweight dynamic CPU autoscaling techniques for short-lived serverless functions, demonstrating their feasibility and effectiveness using Kubernetes and real workloads.

Contribution

It introduces and evaluates tiny autoscalers specifically designed for short-running serverless workloads, filling a gap in existing autoscaling research.

Findings

Dynamic CPU allocation is feasible for serverless functions.

Lightweight autoscaling algorithms perform well with short workloads.

Experimental results show improved resource utilization and reduced throttling.

Abstract

In serverless computing, applications are executed under lightweight virtualization and isolation environments, such as containers or micro virtual machines. Typically, their memory allocation is set by the user before deployment. All other resources, such as CPU, are allocated by the provider statically and proportionally to memory allocations. This contributes to either under-utilization or throttling. The former significantly impacts the provider, while the latter impacts the client. To solve this problem and accommodate both clients and providers, a solution is dynamic CPU allocation achieved through autoscaling. Autoscaling has been investigated for long-running applications using history-based techniques and prediction. However, serverless applications are short-running workloads, where such techniques are not well suited. In this paper, we investigate tiny autoscalers and how dynamic CPU allocation techniques perform for short-running serverless workloads. We experiment with Kubernetes as the underlying platform and implement using its vertical pod autoscaler several dynamic CPU rightsizing techniques. We compare these techniques using state-of-the-art serverless workloads. Our experiments show that dynamic CPU allocation for short-running serverless functions is feasible and can be achieved with lightweight algorithms that offer good performance.