Characterizing Commodity Serverless Computing Platforms

TL;DR

This paper systematically characterizes major commodity serverless computing platforms through qualitative and quantitative analyses, providing insights into their performance, resource efficiency, and operational differences to aid developers and vendors.

Contribution

It offers the first comprehensive comparison of AWS Lambda, Google Cloud Functions, Azure Functions, and Alibaba Cloud Function Compute using both documentation-based taxonomy and performance benchmarks.

Findings

Platform startup latency varies significantly across providers.

Resource efficiency differs notably among platforms based on benchmark tests.

Concurrency handling performance shows distinct patterns with potential causes analyzed.

Abstract

Serverless computing has become a new trending paradigm in cloud computing, allowing developers to focus on the development of core application logic and rapidly construct the prototype via the composition of independent functions. With the development and prosperity of serverless computing, major cloud vendors have successively rolled out their commodity serverless computing platforms. However, the characteristics of these platforms have not been systematically studied. Measuring these characteristics can help developers to select the most adequate serverless computing platform and develop their serverless-based applications in the right way. To fill this knowledge gap, we present a comprehensive study on characterizing mainstream commodity serverless computing platforms, including AWS Lambda, Google Cloud Functions, Azure Functions, and Alibaba Cloud Function Compute. Specifically, we conduct both qualitative analysis and quantitative analysis. In qualitative analysis, we compare these platforms from three aspects (i.e., development, deployment, and runtime) based on their official documentation to construct a taxonomy of characteristics. In quantitative analysis, we analyze the runtime performance of these platforms from multiple dimensions with well-designed benchmarks. First, we analyze three key factors that can influence the startup latency of serverless-based applications. Second, we compare the resource efficiency of different platforms with 16 representative benchmarks. Finally, we measure their performance difference when dealing with different concurrent requests, and explore the potential causes in a black-box fashion. Based on the results of both qualitative and quantitative analysis, we derive a series of findings and provide insightful implications for both developers and cloud vendors.