Machine learning for ultrafast X-ray diffraction patterns on large-scale GPU clusters

TL;DR

This paper presents a distributed GPU-based machine learning algorithm for real-time processing of large-scale X-ray diffraction data, enabling rapid 3D imaging of complex molecules.

Contribution

It introduces a highly efficient, scalable algorithm for inverting large diffraction datasets on GPU clusters, facilitating real-time molecular imaging.

Findings

Demonstrates effective scaling on hundreds of GPUs

Validates approach with real and synthetic data

Discusses computational viability for future large datasets

Abstract

The classical method of determining the atomic structure of complex molecules by analyzing diffraction patterns is currently undergoing drastic developments. Modern techniques for producing extremely bright and coherent X-ray lasers allow a beam of streaming particles to be intercepted and hit by an ultrashort high energy X-ray beam. Through machine learning methods the data thus collected can be transformed into a three-dimensional volumetric intensity map of the particle itself. The computational complexity associated with this problem is very high such that clusters of data parallel accelerators are required. We have implemented a distributed and highly efficient algorithm for inversion of large collections of diffraction patterns targeting clusters of hundreds of GPUs. With the expected enormous amount of diffraction data to be produced in the foreseeable future, this is the required scale to approach real time processing of data at the beam site. Using both real and synthetic data we look at the scaling properties of the application and discuss the overall computational viability of this exciting and novel imaging technique.