Highest-performance Stream Processing

TL;DR

This paper introduces a high-performance stream processing library for OCaml that combines ease of use with near hand-written speed and memory efficiency, supporting a wide range of stream operations.

Contribution

It presents a novel OCaml streaming library that achieves top performance through assured code generation and complete fusion, surpassing existing libraries.

Findings

Achieves the highest performance among OCaml streaming libraries

Attains speed and memory efficiency comparable to hand-written state machines

Supports a comprehensive set of stream operations with a declarative interface

Abstract

We present the stream processing library that achieves the highest performance of existing OCaml streaming libraries, attaining the speed and memory efficiency of hand-written state machines. It supports finite and infinite streams with the familiar declarative interface, of any combination of map, filter, take(while), drop(while), zip, flatmap combinators and tupling. Experienced users may use the lower-level interface of stateful streams and implement accumulating maps, compression and windowing. The library is based on assured code generation (at present, of OCaml and C) and guarantees in all cases complete fusion.