BeFaaS: An Application-Centric Benchmarking Framework for FaaS Platforms

TL;DR

BeFaaS is an extensible, application-centric benchmarking framework for FaaS platforms that evaluates real-world workloads, supports multi-provider setups, and offers detailed performance insights.

Contribution

It introduces a comprehensive benchmarking framework that goes beyond microbenchmarks to evaluate FaaS platforms with realistic applications and multi-provider support.

Findings

Successfully benchmarks three major FaaS providers

Automates benchmark execution with minimal configuration

Provides detailed insights into FaaS platform performance

Abstract

Following the increasing interest and adoption of FaaS systems, benchmarking frameworks for determining non-functional properties have also emerged. While existing (microbenchmark) frameworks only evaluate single aspects of FaaS platforms, a more holistic, application-driven approach is still missing. In this paper, we design and present BeFaaS, an extensible application-centric benchmarking framework for FaaS environments that focuses on the evaluation of FaaS platforms through realistic and typical examples of FaaS applications. BeFaaS includes a built-in e-commerce benchmark, is extensible for new workload profiles and new platforms, supports federated benchmark runs in which the benchmark application is distributed over multiple providers, and supports a fine-grained result analysis. Our evaluation compares three major FaaS providers in single cloud provider setups and shows that BeFaaS is capable of running each benchmark automatically with minimal configuration effort and providing detailed insights for each interaction.