Testing the data framework for an AI algorithm in preparation for high data rate X-ray facilities

TL;DR

This paper presents a data framework for high-throughput X-ray experiments at next-generation facilities, demonstrating real-time data capture and machine learning analysis at MHz rates to enable rapid scientific insights.

Contribution

The study introduces a novel data handling and analysis framework capable of real-time processing of single-shot X-ray data at MHz repetition rates, suitable for next-generation X-ray free electron lasers.

Findings

Framework successfully captures single-shot data at high rates

Machine learning algorithm extracts contrast parameters automatically

Feasibility of live data analysis at MHz rates demonstrated

Abstract

The advent of next-generation X-ray free electron lasers will be capable of delivering X-rays at a repetition rate approaching 1 MHz continuously. This will require the development of data systems to handle experiments at these type of facilities, especially for high throughput applications, such as femtosecond X-ray crystallography and X-ray photon fluctuation spectroscopy. Here, we demonstrate a framework which captures single shot X-ray data at the LCLS and implements a machine-learning algorithm to automatically extract the contrast parameter from the collected data. We measure the time required to return the results and assess the feasibility of using this framework at high data volume. We use this experiment to determine the feasibility of solutions for `live' data analysis at the MHz repetition rate.