A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems
Jingqiao Tang, Ryan Bausback, Feng Bao, Guannan Zhang, Phuoc-Toan Huynh

TL;DR
This paper presents a novel data assimilation framework using an ensemble score filter to enhance machine learning models for long-term forecasting of complex dynamical systems, reducing predictive uncertainty.
Contribution
It introduces a hybrid framework combining ML with the Ensemble Score Filter for improved long-term predictions in high-dimensional nonlinear systems.
Findings
Effective reduction of predictive uncertainty in Lorenz-96 system
Improved long-term forecasting accuracy for KdV equation
Demonstrated robustness of EnSF-enhanced ML models
Abstract
We introduce a score-filter-enhanced data assimilation framework designed to reduce predictive uncertainty in machine learning (ML) models for data-driven dynamical system forecasting. Machine learning serves as an efficient numerical model for predicting dynamical systems. However, even with sufficient data, model uncertainty remains and accumulates over time, causing the long-term performance of ML models to deteriorate. To overcome this difficulty, we integrate data assimilation techniques into the training process to iteratively refine the model predictions by incorporating observational information. Specifically, we apply the Ensemble Score Filter (EnSF), a generative AI-based training-free diffusion model approach, for solving the data assimilation problem in high-dimensional nonlinear complex systems. This leads to a hybrid data assimilation-training framework that combines ML…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Meteorological Phenomena and Simulations · Neural Networks and Reservoir Computing
