Extreme data compression while searching for new physics

TL;DR

This paper introduces a novel data compression method that preserves the ability to detect and analyze new physics in high-dimensional datasets, enabling rapid, precise constraints on both standard and non-standard models.

Contribution

It extends the MOPED algorithm with a generalized PCA approach, creating MOPED-PC, to efficiently scout for new physics during data compression without losing sensitivity.

Findings

Enables analytic Bayesian evidence computation for new physics detection.

Provides fast estimates of non-standard model parameters.

Preserves standard model parameter precision when no new physics is present.

Abstract

Bringing a high-dimensional dataset into science-ready shape is a formidable challenge that often necessitates data compression. Compression has accordingly become a key consideration for contemporary cosmology, affecting public data releases, and reanalyses searching for new physics. However, data compression optimized for a particular model can suppress signs of new physics, or even remove them altogether. We therefore provide a solution for exploring new physics \emph{during} data compression. In particular, we store additional agnostic compressed data points, selected to enable precise constraints of non-standard physics at a later date. Our procedure is based on the maximal compression of the MOPED algorithm, which optimally filters the data with respect to a baseline model. We select additional filters, based on a generalised principal component analysis, which are carefully constructed to scout for new physics at high precision and speed. We refer to the augmented set of filters as MOPED-PC. They enable an analytic computation of Bayesian evidences that may indicate the presence of new physics, and fast analytic estimates of best-fitting parameters when adopting a specific non-standard theory, without further expensive MCMC analysis. As there may be large numbers of non-standard theories, the speed of the method becomes essential. Should no new physics be found, then our approach preserves the precision of the standard parameters. As a result, we achieve very rapid and maximally precise constraints of standard and non-standard physics, with a technique that scales well to large dimensional datasets.