# Overbooking Microservices in the Cloud

## TL;DR

This paper proposes a method for overbooking serverless and microservice workloads in cloud environments, optimizing resource utilization while considering demand constraints and execution time distributions.

## Contribution

It extends bounds for overbooking strategies from single to multiple service tiers and applies to both public serverless and private microservice scheduling.

## Key findings

- Extended bounds for multi-tier overbooking.
- Framework for resource estimation in private microservices.
- Potential use of Markov inequality for dependent job times.

## Abstract

We consider the problem of scheduling serverless-computing instances such as Amazon Lambda functions, or scheduling microservices within (privately held) virtual machines (VMs). Instead of a quota per tenant/customer, we assume demand for Lambda functions is modulated by token-bucket mechanisms per tenant. Such quotas are due to, e.g., limited resources (as in a fog/edge-cloud context) or to prevent excessive unauthorized invocation of numerous instances by malware. Based on an upper bound on the stationary number of active "Lambda servers" considering the execution-time distribution of Lambda functions, we describe an approach that the cloud could use to overbook Lambda functions for improved utilization of IT resources. An earlier bound for a single service tier is extended to multiple service tiers. For the context of scheduling microservices in a private setting, the framework could be used to determine the required VM resources for a token-bucket constrained workload stream. Finally, we note that the looser Markov inequality may be useful in settings where the job service times are dependent.