Universal Workers: A Vision for Eliminating Cold Starts in Serverless Computing

TL;DR

This paper introduces universal workers in serverless computing, aiming to eliminate cold start delays by leveraging workload skewness and caching strategies, thus enabling more scalable and efficient FaaS platforms.

Contribution

It proposes a novel universal worker architecture that reduces cold start latency by exploiting request skewness and implementing a three-tier caching system.

Findings

Workload analysis reveals high request skewness in FaaS platforms.

Universal workers with caching significantly reduce cold start latency.

Approach improves scalability and efficiency of serverless platforms.

Abstract

Serverless computing enables developers to deploy code without managing infrastructure, but suffers from cold start overhead when initializing new function instances. Existing solutions such as "keep-alive" or "pre-warming" are costly and unreliable under bursty workloads. We propose universal workers, which are computational units capable of executing any function with minimal initialization overhead. Based on an analysis of production workload traces, our key insight is that requests in Function-as-a-Service (FaaS) platforms show a highly skewed distribution, with most requests invoking a small subset of functions. We exploit this observation to approximate universal workers through locality groups and three-tier caching (handler, install, import). With this work, we aim to enable more efficient and scalable FaaS platforms capable of handling diverse workloads with minimal initialization overhead.