Interference detection in radio astronomy applying Shapiro-Wilks normality test, spectral entropy, and spectral relative entropy

TL;DR

This paper evaluates statistical measures including the Shapiro-Wilks normality test, spectral entropy, and spectral relative entropy for detecting radio-frequency interference in radio astronomy data, showing improvements over traditional methods.

Contribution

It introduces the use of SW, SE, and SRE measures for RFI detection in radio astronomy, demonstrating their effectiveness compared to baseline methods.

Findings

SE, SRE, SK, and SW improve signal-to-noise ratio.

Algorithms effectively characterize broadband RFI.

Time- and frequency-variable RFI are best detected by SK and SW.

Abstract

Radio-frequency interference (RFI) is becoming an increasingly significant problem for most radio telescopes. Working with Green Bank Telescope data from PSR J1730+0747 in the form of complex-valued channelized voltages and their respective high-resolution power spectral densities, we evaluate a variety of statistical measures to characterize RFI. As a baseline for performance comparison, we use median absolute deviation (MAD) in complex channelized voltage data and spectral kurtosis (SK) in power spectral density data to characterize and filter out RFI. From a new perspective, we implement the Shapiro-Wilks (SW) test for normality and two information theoretical measures, spectral entropy (SE) and spectral relative entropy (SRE), and apply them to mitigate RFI. The baseline RFI mitigation algorithms are compared against our novel RFI detection algorithms to determine how effective and robust the performance is. Except for MAD, we find significant improvements in signal-to-noise ratio through the application of SE, symmetrical SRE, asymmetrical SRE, SK, and SW. These algorithms also do a good job of characterizing broadband RFI. Time- and frequency-variable RFI signals are best detected by SK and SW tests.