HotSwap: Enabling Live Dependency Sharing in Serverless Computing

TL;DR

HotSwap is a provider-side optimization for serverless computing that reduces cold-start latency by sharing pre-warmed dependency images across functions, especially benefiting skewed workloads.

Contribution

It introduces a cross-function pre-warming strategy that enables efficient dependency sharing while respecting cache constraints, improving cold-start times and resource utilization.

Findings

Accelerates dependency loading by 2.2 to 3.2 times on tested functions.

Saves 88% of cache space compared to function-specific pre-baking.

Effective in skewed production workloads with diverse dependencies.

Abstract

This work presents HotSwap, a novel provider-side cold-start optimization for serverless computing. This optimization reduces cold-start time when booting and loading dependencies at runtime inside a function container. Previous research has extensively focused on reducing cold-start latency for specific functions. However, little attention has been given to skewed production workloads. In such cases, cross-function optimization becomes essential. Without cross-function optimization, a cloud provider is left with two equally poor options: (i) Either the cloud provider gives up optimization for each function in the long tail (which is slow); or (ii) the cloud provider applies function-specific optimizations (e.g., cache function images) to every function in the long tail (which violates the vendor's cache constraints). HotSwap demonstrates cross-function optimization using a novel pre-warming strategy. In this strategy, a pre-initialized live dependency image is migrated to the new function instance. At the same time, HotSwap respects the provider's cache constraints, because a single pre-warmed dependency image in the cache can be shared among all serverless functions that require that image. HotSwap has been tested on seven representative functions from FunctionBench. In those tests, HotSwap accelerates dependency loading for those serverless functions with large dependency requirements by a factor ranging from 2.2 to 3.2. Simulation experiments using Azure traces indicate that HotSwap can save 88\% of space, compared with a previous function-specific method, PreBaking, when sharing a dependency image among ten different functions.