Systematic Spectral Distortion from Digital Whitening in Radio Telescopes and Implications for 21 cm Cosmology

TL;DR

Digital whitening and re-quantization in radio telescopes cause subtle spectral distortions that can impact 21 cm cosmology measurements, but mitigation strategies can reduce this effect.

Contribution

This paper identifies a previously unrecognized systematic spectral distortion caused by digital flattening and re-quantization in radio telescopes, with analysis and mitigation methods.

Findings

The distortion affects the gain-vs.-frequency function in radio telescopes.

The effect can be significant for 21 cm cosmology.

Mitigation strategies include adjusting gain distribution and adding dithering.

Abstract

We identify a systematic distortion of the gain-vs.-frequency function of radio telescopes caused by digital flattening ("whitening") of the signal's spectrum followed by re-quantization, a common pair of processes in the signal processing of modern telescopes. Wide-bandwidth telescopes often have a large variation of signal power over frequency. Flattening of the spectrum allows samples of the channelized signal to be represented in a small number of bits, allowing efficient downstream processing. However, we show that this produces subtle systematic error in the measured spectra. We explore this effect in data from the Owens Valley Radio Observatory's Long Wavelength Array (OVRO-LWA) and through detailed semi-analytic simulations. Although the effect can be small so that it has heretofore been unrecognized, we demonstrate that it produces distortion of the spectrum at a level that is problematic for some science, in particular 21 cm cosmology. Finally, we explore mitigation strategies, showing that the effect can be substantially reduced by careful choice of the gain distribution along the signal path or by incorporating dithering in the re-quantization step.