Creating A Galactic Plane Atlas With Amazon Web Services

TL;DR

This paper demonstrates how astronomers can leverage AWS cloud computing to efficiently create a multi-wavelength Galactic Plane atlas, addressing technical challenges and resource management.

Contribution

It provides a practical example of using AWS for large-scale astronomical data processing and details the technical considerations involved.

Findings

Successfully created a 16-wavelength Galactic Plane atlas

Outlined open source tools for cloud-based data management

Discussed resource optimization and cost management strategies

Abstract

This paper describes by example how astronomers can use cloud-computing resources offered by Amazon Web Services (AWS) to create new datasets at scale. We have created from existing surveys an atlas of the Galactic Plane at 16 wavelengths from 1 {\mu}m to 24 {\mu}m with pixels co-registered at spatial sampling of 1 arcsec. We explain how open source tools support management and operation of a virtual cluster on AWS platforms to process data at scale, and describe the technical issues that users will need to consider, such as optimization of resources, resource costs, and management of virtual machine instances.