Using the Parallel Virtual Machine for Everyday Analysis

TL;DR

This paper demonstrates how integrating the Parallel Virtual Machine with the ISIS system enables astronomers to perform routine, parallelized data analysis, significantly reducing computation times for large datasets.

Contribution

It introduces a method to incorporate PVM into ISIS as a scriptable module, facilitating widespread use of parallel computing in astronomical data analysis.

Findings

Distributed calculations over 25+ CPUs reduce execution times dramatically.

The approach applies broadly to various modeling problems in astronomy.

Enhanced transparency and usability for routine data analysis.

Abstract

A review of the literature reveals that while parallel computing is sometimes employed by astronomers for custom, large-scale calculations, no package fosters the routine application of parallel methods to standard problems in astronomical data analysis. This paper describes our attempt to close that gap by wrapping the Parallel Virtual Machine (PVM) as a scriptable S-Lang module. Using PVM within ISIS, the Interactive Spectral Interpretation System, we've distributed a number of representive calculations over a network of 25+ CPUs to achieve dramatic reductions in execution times. We discuss how the approach applies to a wide class of modeling problems, outline our efforts to make it more transparent for common use, and note its growing importance in the context of the large, multi-wavelength datasets used in modern analysis.