Accelerating Serverless Computing by Harvesting Idle Resources

TL;DR

This paper introduces Freyr, a resource manager for serverless platforms that dynamically harvests idle resources from over-provisioned functions to improve efficiency and reduce latency using deep reinforcement learning.

Contribution

Freyr is a novel resource management system that leverages real-time monitoring and deep reinforcement learning to optimize resource utilization in serverless computing.

Findings

Harvested 38.8% of idle resources during function invocations.

Accelerated 39.2% of invocations with harvested resources.

Reduced 99th-percentile function response latency by 32.1%.

Abstract

Serverless computing automates fine-grained resource scaling and simplifies the development and deployment of online services with stateless functions. However, it is still non-trivial for users to allocate appropriate resources due to various function types, dependencies, and input sizes. Misconfiguration of resource allocations leaves functions either under-provisioned or over-provisioned and leads to continuous low resource utilization. This paper presents Freyr, a new resource manager (RM) for serverless platforms that maximizes resource efficiency by dynamically harvesting idle resources from over-provisioned functions to under-provisioned functions. Freyr monitors each function's resource utilization in real-time, detects over-provisioning and under-provisioning, and learns to harvest idle resources safely and accelerates functions efficiently by applying deep reinforcement learning algorithms along with a safeguard mechanism. We have implemented and deployed a Freyr prototype in a 13-node Apache OpenWhisk cluster. Experimental results show that 38.8% of function invocations have idle resources harvested by Freyr, and 39.2% of invocations are accelerated by the harvested resources. Freyr reduces the 99th-percentile function response latency by 32.1% compared to the baseline RMs.