An algorithm for the automatic deglitching of x-ray absorption spectroscopy data

TL;DR

This paper introduces a robust, automated Python algorithm for deglitching x-ray absorption spectroscopy data, effectively removing artifacts across large datasets using statistical methods.

Contribution

The paper presents a novel, statistically robust algorithm combining Savitzky-Golay filtering and the generalized extreme Studentized deviate test for automatic deglitching of XAS data.

Findings

Effective removal of glitches from large XAS datasets

Robust and repeatable glitch detection and removal

Applicable to various spectra without manual intervention

Abstract

Analysis of x-ray absorption spectroscopy (XAS) data often involves the removal of artifacts or glitches from the acquired signal, a process commonly known as deglitching. Glitches result either from specific orientations of monochromator crystals or from scattering by crystallites in the sample itself. Since the precise energy or wavelength location and the intensity of glitches in a spectrum cannot always be predicted, deglitching is often performed on a per spectrum basis by the analyst. Some routines have been proposed, but they are prone to arbitrary selection of spectral artifacts and are often inadequate for processing large data sets. Here we present a statistically robust algorithm, implemented as a Python program, for the automatic detection and removal of glitches that can be applied to a large number of spectra. It uses a Savitzky-Golay filter to smooth spectra and the generalized extreme Studentized deviate test to identify outliers. We achieve robust, repeatable, and selective removal of glitches using this algorithm.