Serverless Abstractions for Short-Running, Lightweight Streams

TL;DR

This paper introduces stream functions, a novel serverless abstraction optimized for short, lightweight, and unpredictable streams, achieving significant overhead reduction and maintaining elasticity for stateful stream processing.

Contribution

It proposes stream functions as an extension of the FaaS model, enabling efficient, stateful, short-stream processing with minimal overhead and seamless integration with serverless platforms.

Findings

Reduces processing overhead by ~99% compared to traditional stream processing engines.

Provides performance comparable to standard serverless functions with stream semantics.

Enables efficient stateful processing for short, lightweight streams.

Abstract

Serverless computing and stream processing represent two dominant paradigms for event-driven data processing, yet both make assumptions that render them inefficient for short-running, lightweight, and unpredictable streams that require stateful processing. We propose stream functions as a novel extension of the Function-as-a-Serivce model that treat short streams as the unit of execution, state, and scaling. Stream functions process streams via an iterator-based interface, enabling seamless inter-event logic while retaining the elasticity and scale-to-zero capabilities offered by serverless platforms. Our evaluation shows that stream functions reduce the processing overhead by ~99 % compared to a mature stream process- ing engine in a video-processing use case. By providing comparable performance to serverless functions with stream semantics, stream functions provide an effective and efficient abstractions for a class of workloads underserved by existing models.