Data processing pipeline for Herschel HIFI

TL;DR

This paper describes the development and implementation of a modular data processing pipeline for the Herschel HIFI instrument, enabling robust calibration and analysis of over 9100 astronomical observations during the mission.

Contribution

The paper introduces a flexible, modular software pipeline that processed all HIFI data, supporting both automated and interactive analysis, and ensuring consistent, reliable calibration throughout the mission.

Findings

Processed all HIFI observations within the Herschel environment

Enabled reprocessing by the astronomical community

Provided automated quality reports and consistency checks

Abstract

{Context}. The HIFI instrument on the Herschel Space Observatory performed over 9100 astronomical observations, almost 900 of which were calibration observations in the course of the nearly four-year Herschel mission. The data from each observation had to be converted from raw telemetry into calibrated products and were included in the Herschel Science Archive. {Aims}. The HIFI pipeline was designed to provide robust conversion from raw telemetry into calibrated data throughout all phases of the HIFI missions. Pre-launch laboratory testing was supported as were routine mission operations. {Methods}. A modular software design allowed components to be easily added, removed, amended and/or extended as the understanding of the HIFI data developed during and after mission operations. {Results}. The HIFI pipeline processed data from all HIFI observing modes within the Herschel automated processing environment as well as within an interactive environment. The same software can be used by the general astronomical community to reprocess any standard HIFI observation. The pipeline also recorded the consistency of processing results and provided automated quality reports. Many pipeline modules were in use since the HIFI pre-launch instrument level testing. {Conclusions}. Processing in steps facilitated data analysis to discover and address instrument artefacts and uncertainties. The availability of the same pipeline components from pre-launch throughout the mission made for well-understood, tested, and stable processing. A smooth transition from one phase to the next significantly enhanced processing reliability and robustness.