# AdiosStMan: Parallelizing Casacore Table Data System Using Adaptive IO System

## TL;DR

This paper presents AdiosStMan, a parallel storage manager for the Casacore Table Data System using the Adaptive IO System, significantly improving parallel I/O throughput for large-scale astronomical data processing.

## Contribution

It introduces a novel ADIOS-based parallel storage manager for CTDS, enabling efficient parallel I/O in CASA without altering existing workflows.

## Key findings

- Demonstrated high parallel I/O throughput with ADIOS-based CTDS
- Validated the storage manager using CASA MSTransform tasks
- Achieved scalable performance in massively parallel scenarios

## Abstract

In this paper, we investigate the Casacore Table Data System (CTDS) used in the casacore and CASA libraries, and methods to parallelize it. CTDS provides a storage manager plugin mechanism for third-party devel- opers to design and implement their own CTDS storage managers. Hav- ing this in mind, we looked into various storage backend techniques that can possibly enable parallel I/O for CTDS by implementing new storage managers. After carrying on benchmarks showing the excellent parallel I/O throughput of the Adaptive IO System (ADIOS), we implemented an ADIOS based parallel CTDS storage manager. We then applied the CASA MSTransform frequency split task to verify the ADIOS Storage Manager. We also ran a series of performance tests to examine the I/O throughput in a massively parallel scenario.