PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift

TL;DR

PIMSM introduces a physics-informed multi-scale neural architecture that enhances stability and transferability of scientific models across distribution shifts by aligning with natural physical timescales.

Contribution

It formalizes the concept of temporal kernel mismatch and proposes PIMSM, a novel state-space model that incorporates physical timescales for improved robustness.

Findings

Improves robustness under temporal truncation and low-resource transfer.

Achieves lowest MAE in weather forecasting across out-of-distribution stations.

Enhances stability in fMRI analysis under various temporal conditions.

Abstract

Scientific foundation models are expected to reuse representations under changes in dataset, acquisition protocol, and deployment domain, yet many sequence backbones treat scientific temporal structure as an unconstrained pattern to be fitted. We argue that this misses a central property of natural dynamical systems: neural and atmospheric time series are organized by interacting processes across multiple physical timescales, and failure to preserve this multiscale structure contributes to brittleness under distribution shift. We formalize this failure mode as temporal kernel mismatch, where a model fits in-distribution dynamics with an effective memory policy that is not anchored to the signal's physical timescales, leading to representation drift and degraded transfer. We propose Physics-Informed Multi-Scale Mamba (PIMSM), a state-space architecture that maps spectrum-estimated transition points between frequency regimes (knee frequencies) to scale-specific discretization parameters and anchors them to acquisition time units. On Human Connectome Project fMRI, PIMSM improves robustness and representation stability under severe temporal-context truncation, extreme low-resource transfer, and resting-state-to-task-state generalization. Without modality-specific adaptation, the same architecture also attains the lowest variable-wise MAE across all reported horizons and variables on Weather-5K held-out-station spatial out-of-distribution forecasting. These results support temporal-scale alignment as a practical inductive bias for scientific foundation models that must preserve structure, not only fit correlations, under deployment shift.