Galactic Double Neutron Star total masses and Gaussian mixture model selection

TL;DR

This paper re-analyzes galactic double neutron star total mass data using various statistical tests, finding no strong evidence for multiple sub-populations beyond a single Gaussian distribution.

Contribution

It demonstrates that model selection for small datasets should incorporate penalization and multiple criteria, challenging previous claims of sub-populations based solely on likelihood ratios.

Findings

No robust preference for two sub-populations over one

Model selection criteria favor simpler models with fewer parameters

Additional data are needed to confirm the existence of sub-populations

Abstract

Huang et al. [arXiv:1804.03101] have analysed the population of 15 known galactic Double Neutron Stars (DNSs) regarding the total masses of these systems. They suggest the existence of two sub-populations, and report likelihood-based preference for a two-component Gaussian mixture model over a single Gaussian distribution. This note offers a cautionary perspective on model selection for this data set: Especially for such a small sample size, a pure likelihood ratio test can encourage overfitting. This can be avoided by penalising models with a higher number of free parameters. Re-examining the DNS total mass data set within the class of Gaussian mixture models, this can be achieved through several simple and well-established statistical tests, including information criteria (AICc, BIC), cross-validation, Bayesian evidence ratios and a penalised EM-test. While this re-analysis confirms the basic finding that a two-component mixture is consistent with the data, the model selection criteria consistently indicate that there is no robust preference for it over a single-component fit. Additional DNS discoveries will be needed to settle the question of sub-populations.