Making Serverless Computing Extensible: A Case Study of Serverless Data Analytics

TL;DR

This paper introduces an extensible serverless computing framework called Proteus, enabling domain-specific optimizations for data analytics while maintaining simplicity and flexibility for developers.

Contribution

It proposes a novel design principle for extensibility in serverless platforms and implements it in Proteus with a new decision workflow abstraction.

Findings

Proteus improves analytical query execution performance.

Supports fine-grained resource sharing across applications.

Demonstrates effective optimization in data analytics workloads.

Abstract

Serverless computing has attracted a broad range of applications due to its ease of use and resource elasticity. However, developing serverless applications often poses a dilemma -- relying on general-purpose serverless platforms can fall short of delivering satisfactory performance for complex workloads, whereas building application-specific serverless systems undermines the simplicity and generality. In this paper, we propose an extensible design principle for serverless computing. We argue that a platform should enable developers to extend system behaviors for domain-specialized optimizations while retaining a shared, easy-to-use serverless environment. We take data analytics as a representative serverless use case and realize this design principle in Proteus. Proteus introduces a novel abstraction of decision workflows, allowing developers to customize control-plane behaviors for improved application performance. Preliminary results show that Proteus's prototype effectively optimizes analytical query execution and supports fine-grained resource sharing across diverse applications.