SeBS-Flow: Benchmarking Serverless Cloud Function Workflows

TL;DR

SeBS-Flow is a comprehensive, platform-agnostic benchmarking suite for serverless workflows, enabling consistent performance evaluation across different cloud platforms with real-world and microbenchmark applications.

Contribution

It introduces the first open-source benchmarking suite for serverless workflows, facilitating fair and consistent performance comparisons across platforms.

Findings

Performance varies significantly across platforms.

Cost and scalability metrics differ among cloud providers.

SeBS-Flow enables rigorous, comparable evaluations over time.

Abstract

Serverless computing has emerged as a prominent paradigm, with a significant adoption rate among cloud customers. While this model offers advantages such as abstraction from the deployment and resource scheduling, it also poses limitations in handling complex use cases due to the restricted nature of individual functions. Serverless workflows address this limitation by orchestrating multiple functions into a cohesive application. However, existing serverless workflow platforms exhibit significant differences in their programming models and infrastructure, making fair and consistent performance evaluations difficult in practice. To address this gap, we propose the first serverless workflow benchmarking suite SeBS-Flow, providing a platform-agnostic workflow model that enables consistent benchmarking across various platforms. SeBS-Flow includes six real-world application benchmarks and four microbenchmarks representing different computational patterns. We conduct comprehensive evaluations on three major cloud platforms, assessing performance, cost, scalability, and runtime deviations. We make our benchmark suite open-source, enabling rigorous and comparable evaluations of serverless workflows over time.