SimFaaS: A Performance Simulator for Serverless Computing Platforms

TL;DR

SimFaaS is an open-source simulation platform designed to accurately model and predict performance metrics of serverless computing platforms, aiding developers and providers in optimization and design evaluation.

Contribution

This paper introduces SimFaaS, a comprehensive, extendable simulation tool for serverless platforms, filling the gap of lacking simulation frameworks and enabling cost-effective performance analysis.

Findings

SimFaaS accurately predicts response time and cold start probability.

The platform helps optimize serverless applications for cost and performance.

Validation against AWS Lambda traces shows high prediction accuracy.

Abstract

Developing accurate and extendable performance models for serverless platforms, aka Function-as-a-Service (FaaS) platforms, is a very challenging task. Also, implementation and experimentation on real serverless platforms is both costly and time-consuming. However, at the moment, there is no comprehensive simulation tool or framework to be used instead of the real platform. As a result, in this paper, we fill this gap by proposing a simulation platform, called SimFaaS, which assists serverless application developers to develop optimized Function-as-a-Service applications in terms of cost and performance. On the other hand, SimFaaS can be leveraged by FaaS providers to tailor their platforms to be workload-aware so that they can increase profit and quality of service at the same time. Also, serverless platform providers can evaluate new designs, implementations, and deployments on SimFaaS in a timely and cost-efficient manner. SimFaaS is open-source, well-documented, and publicly available, making it easily usable and extendable to incorporate more use case scenarios in the future. Besides, it provides performance engineers with a set of tools that can calculate several characteristics of serverless platform internal states, which is otherwise hard (mostly impossible) to extract from real platforms. We show how SimFaaS facilitates the prediction of essential performance metrics such as average response time, probability of cold start, and the average number of instances reflecting the infrastructure cost incurred by the serverless computing provider. We evaluate the accuracy and applicability of SimFaaS by comparing the prediction results with real-world traces from Amazon AWS Lambda.