A Formalization for Specifying and Implementing Correct Pull-Stream Modules

TL;DR

This paper formalizes the pull-stream design pattern in JavaScript, providing a protocol specification to improve correctness, interoperability, and understanding of modules in streaming applications.

Contribution

It introduces a formal event-based protocol language and precise definitions for pull-stream modules, enabling better reasoning and validation of implementations.

Findings

Validated existing modules against the formal protocol

Identified unspecified behaviors in current modules

Enhanced understanding of pull-stream interactions

Abstract

Pull-stream is a JavaScript demand-driven functional design pattern based on callback functions that enables the creation and easy composition of independent modules that are used to create streaming applications. It is used in popular open source projects and the community around it has created over a hundred compatible modules. While the description of the pull-stream design pattern may seem simple, it does exhibit complicated termination cases. Despite the popularity and large uptake of the pull-stream design pattern, there was no existing formal specification that could help programmers reason about the correctness of their implementations. Thus, the main contribution of this paper is to provide a formalization for specifying and implementing correct pull-stream modules based on the following: (1) we show the pull-stream design pattern is a form of declarative concurrent programming; (2) we present an event-based protocol language that supports our formalization, independently of JavaScript; (3) we provide the first precise and explicit definition of the expected sequences of events that happen at the interface of two modules, which we call the pull-stream protocol; (4) we specify reference modules that exhibit the full range of behaviors of the pull-stream protocol; (5) we validate our definitions against the community expectations by testing the existing core pull-stream modules against them and identify unspecified behaviors in existing modules. Our approach helps to better understand the pull-stream protocol, to ensure interoperability of community modules, and to concisely and precisely specify new pull-stream abstractions in papers and documentation.