Redis-Based Streaming Architecture for Accelerator Beam Instrumentation DAQ Systems

TL;DR

This paper presents a Redis Adapter-based streaming architecture for accelerator beam instrumentation DAQ systems that enhances performance, modularity, and scalability by replacing legacy protocols with Redis and containerization.

Contribution

It introduces the Redis Adapter as a key component for high-performance, modular communication in DAQ systems, enabling legacy compatibility and future scalability.

Findings

Achieved high throughput and reduced latency in data transfer.

Enabled seamless integration of legacy and modern digitizers.

Demonstrated improved system flexibility and modularity.

Abstract

The Fermilab Acceleraor Division, Beam Instrumentation Department, is always adopting modern and current software methodologies for complex DAQ architectures. This paper highlights the Redis Adapter (RA) as the key software component enabling high performance, modular communication between digitizers and distributed control systems by leveraging Redis and containerization. The RA provides a unified, efficient interface between Redis based data streams and consumer systems. In the legacy architecture, digitized data flowed through the custom, UDP based Distributed Data Communication Protocol in the middle layer. In the current system, DDCP remains the ingestion path, while the RA serves as the decoupling layer. The proposed system replaces old VME digitizers with a SOM-based digitizer that communicates with Redis using the RA. The RA acts as both a performance-critical bridge and a protocol-agnostic adapter, ensuring compatibility with legacy control frameworks while enabling future scalability and modularity. This restructuring of the middle layer also helps the system achieve high throughput, reduce latency, and simplify the data path. Finally, we will demonstrate how RA is utilized in our two core products to deliver both legacy compatibility and future flexibility.