COCOA: Cold Start Aware Capacity Planning for Function-as-a-Service Platforms

TL;DR

This paper introduces COCOA, a queueing-based method for capacity planning in FaaS platforms that effectively accounts for cold start delays, reducing over-provisioning while meeting service-level agreements.

Contribution

COCOA is a novel approach that models cold start effects in FaaS, improving capacity planning accuracy over TTL cache-based solutions.

Findings

Reduces over-provisioning by over 70%.

Effectively accounts for cold start delays in capacity planning.

Maintains service-level agreements with optimized resource allocation.

Abstract

Function-as-a-Service (FaaS) is increasingly popular in the software industry due to the implied cost-savings in event-driven workloads and its synergy with DevOps. To size an on-premise FaaS platform, it is important to estimate the required CPU and memory capacity to serve the expected loads. Given the service-level agreements, it is however challenging to take the cold start issue into account during the sizing process. We have investigated the similarity of this problem with the hit rate improvement problem in TTL caches and concluded that solutions for TTL cache, although potentially applicable, lead to over-provisioning in FaaS. Thus, we propose a novel approach, COCOA, to solve this issue. COCOA uses a queueing-based approach to assess the effect of cold starts on FaaS response times. It also considers different memory consumption values depending on whether the function is idle or in execution. Using an event-driven FaaS simulator, FaasSim, we have developed, we show that COCOA can reduce over-provisioning by over 70% in some workloads, while satisfying the service-level agreements.