Delivering SKA Science

TL;DR

The paper discusses the massive data challenges of SKA science, emphasizing the need for distributed, cloud-based solutions to manage costs and ensure effective global scientific collaboration.

Contribution

It analyzes SKA data flow and costs, proposing a shift towards distributed, cloud-enabled data systems inspired by other large-scale scientific projects.

Findings

Survey projects may exceed SKA1 budget without new data management strategies.

Shared resources and costs are essential for sustainable SKA science.

Cloud technologies offer promising solutions for scalable SKA data processing.

Abstract

The SKA will be capable of producing a stream of science data products that are Exa-scale in terms of their storage and processing requirements. This Google-scale enterprise is attracting considerable international interest and excitement from within the industrial and academic communities. In this chapter we examine the data flow, storage and processing requirements of a number of key SKA survey science projects to be executed on the baseline SKA1 configuration. Based on a set of conservative assumptions about trends for HPC and storage costs, and the data flow process within the SKA Observatory, it is apparent that survey projects of the scale proposed will potentially drive construction and operations costs beyond the current anticipated SKA1 budget. This implies a sharing of the resources and costs to deliver SKA science between the community and what is contained within the SKA Observatory. A similar situation was apparent to the designers of the LHC more than 10 years ago. We propose that it is time for the SKA project and community to consider the effort and process needed to design and implement a distributed SKA science data system that leans on the lessons of other projects and looks to recent developments in Cloud technologies to ensure an affordable, effective and global achievement of SKA science goals.