The Observer Effect in World Models: Invasive Adaptation Corrupts Latent Physics

TL;DR

This paper introduces PhyIP, a non-invasive method to evaluate whether neural world models genuinely internalize physical laws by testing linear decodability of physical quantities from frozen representations, revealing that adaptation-based methods can obscure true latent structures.

Contribution

The paper proposes PhyIP, a novel evaluation protocol that assesses physical knowledge in neural models without adaptation, demonstrating its effectiveness across fluid dynamics and orbital mechanics.

Findings

Latent physical quantities are linearly decodable when SSL error is low.

PhyIP recovers physical laws like energy and inverse-square scaling on OOD tests.

Adaptation-based evaluations can obscure true latent physical structures.

Abstract

Determining whether neural models internalize physical laws as world models, rather than exploiting statistical shortcuts, remains challenging, especially under out-of-distribution (OOD) shifts. Standard evaluations often test latent capability via downstream adaptation (e.g., fine-tuning or high-capacity probes), but such interventions can change the representations being measured and thus confound what was learned during self-supervised learning (SSL). We propose a non-invasive evaluation protocol, PhyIP. We test whether physical quantities are linearly decodable from frozen representations, motivated by the linear representation hypothesis. Across fluid dynamics and orbital mechanics, we find that when SSL achieves low error, latent structure becomes linearly accessible. PhyIP recovers internal energy and Newtonian inverse-square scaling on OOD tests (e.g., $\rho > 0.90$). In contrast, adaptation-based evaluations can collapse this structure ($\rho \approx 0.05$). These findings suggest that adaptation-based evaluation can obscure latent structures and that low-capacity probes offer a more accurate evaluation of physical world models.