Toward robust detections of nanohertz gravitational waves

TL;DR

This paper examines the challenges in reliably estimating the background noise for nanohertz gravitational wave detection via pulsar timing arrays, highlighting limitations of current bootstrap methods and proposing alternative approaches.

Contribution

It analyzes the saturation of quasi-independent sky and phase scrambles and discusses methods to increase independent realizations for more reliable significance estimation.

Findings

Sky scrambling saturates after about 10 realizations.

Phase scrambling saturates after about 100 realizations.

Limited independent scrambles hinder reliable high-significance detection.

Abstract

The recent observation of a common red-noise process in pulsar timing arrays (PTAs) suggests that the detection of nanohertz gravitational waves might be around the corner. However, in order to confidently attribute this red process to gravitational waves, one must observe the Hellings-Downs curve -- the telltale angular correlation function associated with a gravitational-wave background. This effort is complicated by the complex modelling of pulsar noise. Without proper care, mis-specified noise models can lead to false-positive detections. Background estimation using bootstrap methods such as sky scrambles and phase shifts, which use the data to characterize the noise, are therefore important tools for assessing significance. We investigate the ability of current PTA experiments to estimate their background with "quasi-independent" scrambles -- characterized by a statistical "match" below the fiducial value: $|M|<0.1$. We show that sky scrambling is affected by "saturation" after $O(10)$ quasi-independent realizations; subsequent scrambles are no longer quasi-independent. We show phase scrambling saturates after $O(100)$ quasi-independent realizations. With so few independent scrambles, it is difficult to make reliable statements about the $\gtrsim 5 \sigma$ tail of the null distribution of the detection statistic. We discuss various methods by which one may increase the number of independent scrambles. We also consider an alternative approach wherein one re-frames the background estimation problem so that the significance is calculated using statistically dependent scrambles. The resulting $p$-value is in principle well-defined but may be susceptible to failure if assumptions about the data are incorrect.