Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems

TL;DR

This paper explains why data-driven models outperform physics-based models in partially-observed systems by using operator theory, statistical physics, and the Koopman operator to unify implicit and explicit modeling approaches.

Contribution

It introduces a novel operator-theoretic framework that unifies data-driven and physics-based models for partially-observed systems, highlighting their implicit and explicit handling of unobserved variables.

Findings

Data-driven models implicitly capture effects of unobserved degrees of freedom.

The framework uses Maximum Entropy and Maximum Caliber measures.

Data-driven models can converge to true observable dynamics.

Abstract

Only a subset of degrees of freedom are typically accessible or measurable in real-world systems. As a consequence, the proper setting for empirical modeling is that of partially-observed systems. Notably, data-driven models consistently outperform physics-based models for systems with few observable degrees of freedom; e.g., hydrological systems. Here, we provide an operator-theoretic explanation for this empirical success. To predict a partially-observed system's future behavior with physics-based models, the missing degrees of freedom must be explicitly accounted for using data assimilation and model parametrization. Data-driven models, in contrast, employ delay-coordinate embeddings and their evolution under the Koopman operator to implicitly model the effects of the missing degrees of freedom. We describe in detail the statistical physics of partial observations underlying data-driven models using novel Maximum Entropy and Maximum Caliber measures. The resulting nonequilibrium Wiener projections applied to the Mori-Zwanzig formalism reveal how data-driven models may converge to the true dynamics of the observable degrees of freedom. Additionally, this framework shows how data-driven models infer the effects of unobserved degrees of freedom implicitly, in much the same way that physics models infer the effects explicitly. This provides a unified implicit-explicit modeling framework for predicting partially-observed systems, with hybrid physics-informed machine learning methods combining implicit and explicit aspects.