Kolmogorov-Sinai entropies identify optimal observables for prediction and dynamics reconstruction in chaotic systems

TL;DR

This paper establishes that Kolmogorov-Sinai entropy effectively predicts the quality of reconstructing chaotic system dynamics from observables, providing a rigorous criterion for optimal observable selection.

Contribution

It proves a theoretical link between Kolmogorov-Sinai entropy and reconstruction error, and empirically validates this on various chaotic systems, enhancing data-driven modeling.

Findings

Kolmogorov-Sinai entropy correlates strongly with reconstruction error.

Theoretical bounds relate Lyapunov exponents and entropy to prediction accuracy.

Empirical validation shows high correlation between entropy estimates and reconstruction quality.

Abstract

Choosing the optimal observable to model dynamical systems for which we do not know the driving equations is nearly always an ad hoc art. Takens' Delay Embedding Theorem guarantees a diffeomorphism between delay-coordinate vectors built from generic scalar observables and the underlying invariant attractor, but is agnostic to optimal observable choice, and formal bounds on reconstruction quality across observables are not known. Here we prove that, under modest technical conditions, the Kolmogorov-Sinai entropy of an observable predicts its reconstruction error of the underlying dynamics in chaotic, ergodic systems. Using the Oseledets Multiplicative Ergodic Theorem, we show that the tangent bundles of reconstructed manifolds admit an invariant Oseledets filtration diffeomorphically related across admissible observables, with Lyapunov exponents controlling the propagation of perturbations. We bound reconstruction error by a quantity monotonically related to the sum of positive Lyapunov exponents and, by the Ruelle inequality, the Kolmogorov-Sinai entropy. We validate this empirically on the Lorenz-63 attractor, the Hastings-Powell food chain, and a tetracosane molecular-dynamics trajectory, recovering Spearman rank correlations between $h^{KS,UB}$ and reconstruction RMSE up to $\rho=+0.89$ ($p=5.5\times 10^{-8}$) for the realistic tetracosane case, sharpening to $\rho=+0.97$ under added measurement noise. This provides a rigorous foundation for observable selection in chaotic systems, applicable as an a priori data-selection criterion for any data-driven modeling pipeline.