Data compression for the First G-APD Cherenkov Telescope

TL;DR

This paper presents a specialized data compression algorithm designed for the high-volume observational data of the FACT Cherenkov Telescope, aiming to improve storage efficiency and data handling speed.

Contribution

It introduces a novel compression algorithm optimized for telescope data, enhancing compression ratio and write speed compared to existing methods.

Findings

Higher compression ratios achieved

Faster data writing performance

Effective handling of large observational datasets

Abstract

The First Geiger-mode Avalanche photodiode (G-APD) Cherenkov Telescope (FACT) has been operating on the Canary island of La Palma since October 2011. Operations were automated so that the system can be operated remotely. Manual interaction is required only when the observation schedule is modified due to weather conditions or in case of unexpected events such as a mechanical failure. Automatic operations enabled high data taking efficiency, which resulted in up to two terabytes of FITS files being recorded nightly and transferred from La Palma to the FACT archive at ISDC in Switzerland. Since long term storage of hundreds of terabytes of observations data is costly, data compression is mandatory. This paper discusses the design choices that were made to increase the compression ratio and speed of writing of the data with respect to existing compression algorithms. Following a more detailed motivation, the FACT compression algorithm along with the associated I/O layer is discussed. Eventually, the performances of the algorithm is compared to other approaches.