A pipeline for Megahertz X-ray Photon Correlation Spectroscopy on soft matter samples at the MID instrument of European XFEL

TL;DR

This paper details a standardized, automated framework for Megahertz X-ray Photon Correlation Spectroscopy on soft matter at EuXFEL, addressing large data challenges and enhancing data quality and accessibility.

Contribution

It introduces a standardized experimental protocol and data processing pipeline for MHz-XPCS at EuXFEL, enabling efficient analysis of large data volumes with minimal user intervention.

Findings

Automated data processing pipeline implemented

Strategies for data reduction and noise improvement proposed

Methods for making data FAIR are discussed

Abstract

In this paper we present the experimental protocol and data processing framework for Megahertz X-ray Photon Correlation Spectroscopy (MHz-XPCS) experiments on soft matter samples, implemented at the Materials Imaging and Dynamics (MID) instrument of the European X-ray Free-Electron Laser (EuXFEL). Due to the introduction of a standard configuration and the implementation of a highly automated data processing pipeline, MHz-XPCS measurements can now be conducted and analyzed with minimal user intervention. A key challenge lies in managing the extremely large data volumes generated by the Adaptive Gain Integrating Pixel Detector (AGIPD) - often reaching several petabytes within a single experiment. We describe the technical implementation, discuss the hardware requirements related to effective parallel data processing, and propose strategies to enhance data quality, in particular related to data reduction strategies and an improvement of the signal-to-noise ratio. Finally, we address strategies for making the processed data FAIR (Findable, Accessible, Interoperable, Reusable), in alignment with the goals of the DAPHNE4NFDI project.