Phase-space entropy at acquisition reflects downstream learnability

TL;DR

This paper introduces a modality-agnostic phase-space entropy measure,  S_{\u00bbi}, that quantifies how acquisition processes preserve or destroy information, predicting downstream learnability across various data modalities without requiring training.

Contribution

It proposes a novel scalar  S_{\u00bbi} based on instrument-resolved phase space to assess information preservation during data acquisition, applicable across multiple modalities.

Findings

 S_{\u00bbi} correctly identifies phase-space coherence and aliasing effects.

It predicts downstream recognition difficulty without training.

Minimizing  S_{\u00bbi} enables optimal sampling design, such as in MRI.

Abstract

Modern learning systems work with data that vary widely across domains, but they all ultimately depend on how much structure is already present in the measurements before any model is trained. This raises a basic question: is there a general, modality-agnostic way to quantify how acquisition itself preserves or destroys the information that downstream learners could use? Here we propose an acquisition-level scalar $\Delta S_{\mathcal B}$ based on instrument-resolved phase space. Unlike pixelwise distortion or purely spectral errors that often saturate under aggressive undersampling, $\Delta S_{\mathcal B}$ directly quantifies how acquisition mixes or removes joint space--frequency structure at the instrument scale. We show theoretically that \(\Delta S_{\mathcal B}\) correctly identifies the phase-space coherence of periodic sampling as the physical source of aliasing, recovering classical sampling-theorem consequences. Empirically, across masked image classification, accelerated MRI, and massive MIMO (including over-the-air measurements), $|\Delta S_{\mathcal B}|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty \emph{without training}. In particular, minimizing $|\Delta S_{\mathcal B}|$ enables zero-training selection of variable-density MRI mask parameters that matches designs tuned by conventional pre-reconstruction criteria. These results suggest that phase-space entropy at acquisition reflects downstream learnability, enabling pre-training selection of candidate sampling policies and as a shared notion of information preservation across modalities.