Posterior Predictive Checks for Gravitational-wave Populations: Limitations and Improvements

TL;DR

This paper evaluates the effectiveness of posterior predictive checks (PPCs) in assessing gravitational-wave population models, highlighting limitations with uncertain parameters and proposing alternative methods, but finds limited improvements with current data.

Contribution

It systematically compares various PPC methods for poorly-constrained parameters and demonstrates their limitations in diagnosing model misspecification in gravitational-wave data.

Findings

PPCs on maximum likelihood parameters are more effective than event-level PPCs.

Alternative PPCs do not significantly improve model diagnostics at current sensitivity.

The GWTC-4.0 catalog shows the Gaussian Spins model misfits in spin magnitude and tilt distributions.

Abstract

When selecting a model to characterize an astrophysical population, it is crucial to assess whether that model fits the data and, if not, how it can be improved. To this end, posterior predictive checks (PPCs) are a widely-used statistical test of model fit when inferring gravitational-wave source populations. However, PPCs exhibit limitations when assessing single-event parameters with large measurement uncertainty, like the spin tilt angles of the binary black holes (BBHs) observable with the LIGO-Virgo-KAGRA (LVK) detectors. When single-event inference is prior-dominated, traditional PPCs fail to flag even very poor model fits. In this work, we assess the efficacy of various alternative PPCs on poorly-constrained parameters. We compare PPCs conducted on event- vs. data-level parameters (e.g. posterior samples vs. maximum likelihood points), and explore two additional event-level PPCs: partial predictive checks and split predictive checks. Independent of measurement uncertainty, we find that PPCs on maximum likelihood parameters are always more discerning of model misspecification than any event-level PPC. However, when investigating simulated GWTC-3.0-like catalogs, none of the alternative PPCs show significant improvement over those traditionally used, indicating that at that sensitivity, any limited information in the data about spin tilts is insufficient to diagnose model misspecification. Finally, we apply our suite of PPCs to the spin magnitude and tilt distributions inferred in the most recent LVK catalog, GWTC-4.0. We conclude that the Gaussian Component Spins model used therein under-predicts BBHs with large spin magnitudes and over-predicts those with perfectly anti-aligned tilts.