End-to-end data acquisition pipeline for the Cherenkov Telescope Array

TL;DR

This paper presents a comprehensive data acquisition pipeline designed for the Cherenkov Telescope Array, capable of handling high data rates, heterogeneous data formats, and providing real-time processing and analysis interfaces.

Contribution

It introduces a novel end-to-end data acquisition pipeline tailored for CTA, including data formatting, camera-specific algorithms, compression, and a Python interface for analysis.

Findings

Successfully handled raw data streams up to 43 Gbps per telescope

Implemented data formatting to a common structure for heterogeneous data

Achieved real-time data processing with current performance metrics

Abstract

The Cherenkov Telescope Array (CTA) will operate several types of telescopes and cameras. The individual camera trigger rates will vary much - from 0.6 to 15 kHz - while the content of the raw data will be heterogeneous. Raw data streams of up to 43 Gbps per telescope must be handled efficiently, from the camera front-ends down to the on-site repository and real-time analysis. In addition, the system must transcode all raw data to a common, pre-calibrated format. We will present the pipeline that we propose to implement this data acquisition pipeline. It will format the raw data to a common structure, provide facilities to run camera-specific algorithms and compress and write data to the on-site repository. We will also present the Python interface that allows the analysis pipeline to access the data. Eventually, the two strategies foreseen to interface the camera servers will be detailed and the current status of the developments for CTA will be given, with the last performance figures measured.