Information theory for dimensionality reduction in dynamical systems

TL;DR

This paper introduces an information-theoretic framework for reducing the complexity of dynamical systems by identifying variables that are most predictive of future states, independent of scale separation.

Contribution

It formalizes a new approach to model reduction based on relevance for prediction, decoupling variable identification from their effective dynamics, and enables data-driven discovery with neural networks.

Findings

Framework effectively identifies relevant variables in chaotic systems.

Method works with real-world data like atmospheric flows and biological videos.

Decoupling variable selection from dynamics simplifies the reduction process.

Abstract

The dynamics of many-body systems can often be captured in terms of only a few relevant variables. Mathematical and numerical approaches exist to identify these variables by exploiting a separation of time scales between slow relevant and fast irrelevant variables, but such a separation of scales is not always obvious or even available. In this work, we introduce an information-theoretic framework for dimensionality reduction in dynamical systems that bypasses this limitation by instead identifying relevant variables based on how predictive they are of the system's future. To do so, we mathematically formalize the intuition that model reduction is about keeping "relevant" information while throwing away "irrelevant" information. We characterize the solution of the resulting optimization problem and prove that it reduces to standard approaches when a separation of time scales is indeed present in the dynamics. Importantly, we find that within this framework, the problems of identifying relevant variables and identifying their effective dynamics decouple and may be solved separately. This makes the method tractable in practice and enables us to derive dimensionally-reduced variables from data with neural networks. Combined with existing equation learning methods, the procedure introduced in this work reveals the dynamical rules governing the system's evolution in a data-driven manner. We illustrate these tools in diverse settings including simulated chaotic systems, uncurated satellite recordings of atmospheric fluid flows, and experimental videos of cyanobacteria colonies in which we discover an emergent synchronization order parameter.