Architecting Complex, Long-Lived Scientific Software

TL;DR

This paper discusses designing and planning long-lived, large-scale scientific software, exemplified by the Square Kilometre Array's data processor, highlighting tailored architecture practices and their role in project approval.

Contribution

It presents a customized architecture analysis process for long-term scientific software, demonstrated through the SKA Observatory project, with insights applicable to similar scientific endeavors.

Findings

Successful architecture plans facilitated project approval.

Customized processes addressed unique project challenges.

Implications for long-lived scientific software architectures.

Abstract

Software is a critical aspect of large-scale science, providing essential capabilities for making scientific discoveries. Large-scale scientific projects are vast in scope, with lifespans measured in decades and costs exceeding hundreds of millions of dollars. Successfully designing software that can exist for that span of time, at that scale, is challenging for even the most capable software companies. Yet scientific endeavors face challenges with funding, staffing, and operate in complex, poorly understood software settings. In this paper we discuss the practice of early-phase software architecture in the Square Kilometre Array Observatory's Science Data Processor. The Science Data Processor is a critical software component in this next-generation radio astronomy instrument. We customized an existing set of processes for software architecture analysis and design to this project's unique circumstances. We report on the series of comprehensive software architecture plans that were the result. The plans were used to obtain construction approval in a critical design review with outside stakeholders. We conclude with implications for other long-lived software architectures in the scientific domain, including potential risks and mitigations.