The Simons Observatory: Deployment of the observatory control system and supporting infrastructure

TL;DR

The paper details the deployment and integration of the observatory control system and supporting infrastructure for the Simons Observatory, enabling coordinated data collection and remote operation of the telescopes and detectors in a high-altitude desert environment.

Contribution

It introduces a distributed control system and deployment strategy using Docker, Ansible, and GitOps for the first time in a large-scale CMB observatory.

Findings

Successful deployment of control and data systems at high altitude

Effective use of containerization and automation tools for infrastructure management

Lessons learned for future improvements in observatory deployment

Abstract

The Simons Observatory (SO) is a cosmic microwave background (CMB) observatory consisting of three small aperture telescopes and one large aperture telescope. SO is located in the Atacama Desert in Chile at an elevation of 5180m. Distributed among the four telescopes are over 60,000 transition-edge sensor (TES) bolometers across six spectral bands centered between 27 and 280 GHz. A large collection of ancillary hardware devices which produce lower rate `housekeeping' data are used to support the detector data collection. We developed a distributed control system, which we call the observatory control system (ocs), to coordinate data collection among all systems within the observatory. ocs is a core component of the deployed site software, interfacing with all on-site hardware. Alongside ocs we utilize a combination of internally and externally developed open source projects to enable remote monitoring, data management, observation coordination, and data processing. Deployment of a majority of the software is done using Docker containers. The deployment of software packages is partially done via automated Ansible scripts, utilizing a GitOps based approach for updating infrastructure on site. We describe an overview of the software and computing systems deployed within SO, including how those systems are deployed and interact with each other. We also discuss the timing distribution system and its configuration as well as lessons learned during the deployment process and where we plan to make future improvements.