Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identification

TL;DR

This paper demonstrates that predicting the system's state directly, rather than velocity or noise, enhances the stability and accuracy of diffusion models in nonlinear spatiotemporal system identification, especially in turbulent regimes.

Contribution

It introduces a simple patch-based transformer approach that emphasizes state prediction, showing improved performance over traditional target parameterizations in turbulent flow modeling.

Findings

State prediction improves rollout stability in turbulent flow simulations.

Direct state prediction reduces long-horizon errors compared to velocity or noise objectives.

Advantages increase with higher per-token dimensionality.

Abstract

Machine learning is becoming increasingly important for nonlinear system identification, including dynamical systems with spatially distributed outputs. However, classical identification and forecasting approaches become markedly less reliable in turbulent-flow regimes, where the dynamics are high-dimensional, strongly nonlinear, and highly sensitive to compounding rollout errors. Diffusion-based models have recently shown improved robustness in this setting and offer probabilistic inference capabilities, but many current implementations inherit target parameterizations from image generation, most commonly noise or velocity prediction. In this work, we revisit this design choice in the context of nonlinear spatiotemporal system identification. We consider a simple, self-contained patch-based transformer that operates directly on physical fields and use turbulent flow simulation as a representative testbed. Our results show that clean-state prediction consistently improves rollout stability and reduces long-horizon error relative to velocity- and noise-based objectives, with the advantage becoming more pronounced as the per-token dimensionality increases. These findings identify target parameterization as a key modeling choice in diffusion-based identification of nonlinear systems with spatial outputs in turbulent regimes.