TensorFlow Chaotic Prediction and Blow Up

TL;DR

This paper explores the use of TensorFlow for predicting chaotic system dynamics, demonstrating short-term success but revealing long-term prediction blow-up due to library nondeterminism.

Contribution

It introduces a method for predicting high-dimensional chaotic systems with neural networks and uncovers an unexpected instability in TensorFlow's long-term predictions.

Findings

Short-term predictions are accurate for chaotic systems.

Long-term predictions deteriorate and blow up over time.

TensorFlow's nondeterministic behavior affects prediction stability.

Abstract

Predicting the dynamics of chaotic systems is one of the most challenging tasks for neural networks, and machine learning in general. Here we aim to predict the spatiotemporal chaotic dynamics of a high-dimensional non-linear system. In our attempt we use the TensorFlow library, representing the state of the art for deep neural networks training and prediction. While our results are encouraging, and show that the dynamics of the considered system can be predicted for short time, we also indirectly discovered an unexpected and undesirable behavior of the TensorFlow library. More specifically, the longer term prediction of the system's chaotic behavior quickly deteriorates and blows up due to the nondeterministic behavior of the TensorFlow library. Here we provide numerical evidence of the short time prediction ability, and of the longer term predictability blow up.