Mitigating Cold Starts in Serverless Platforms: A Pool-Based Approach

TL;DR

This paper presents a pool-based approach to reduce cold start latency in serverless platforms, specifically Knative Serving, achieving significant improvements in response time by maintaining a pool of warm containers.

Contribution

The paper introduces a novel implementation of a warm container pool for Knative Serving, significantly reducing cold start latency compared to the original system.

Findings

85% reduction in P99 response time with the pool-based approach

Effective mitigation of cold start latency in serverless platforms

Demonstrated improvements in latency for bursty workloads

Abstract

Rapid adoption of the serverless (or Function-as-a-Service, FaaS) paradigm, pioneered by Amazon with AWS Lambda and followed by numerous commercial offerings and open source projects, introduces new challenges in designing the cloud infrastructure, balancing between performance and cost. While instant per-request elasticity that FaaS platforms typically offer application developers makes it possible to achieve high performance of bursty workloads without over-provisioning, such elasticity often involves extra latency associated with on-demand provisioning of individual runtime containers that serve the functions. This phenomenon is often called cold starts, as opposed to the situation when a function is served by a pre-provisioned "warm" container, ready to serve requests with close to zero overhead. Providers are constantly working on techniques aimed at reducing cold starts. A common approach to reduce cold starts is to maintain a pool of warm containers, in anticipation of future requests. In this report, we address the cold start problem in serverless architectures, specifically under the Knative Serving FaaS platform. We describe our implementation leveraging a pool of function instances, and evaluate the latency compared to the original implementation, resulting in a 85% reduction of P99 response time for a single instance pool.