BAMCAFE: A Bayesian Machine Learning Advanced Forecast Ensemble Method for Complex Turbulent Systems with Partial Observations

TL;DR

BAMCAFE is a Bayesian machine learning ensemble method that combines physics-informed models with data assimilation to improve forecasts of complex turbulent systems, especially under partial and noisy observations.

Contribution

This paper introduces BAMCAFE, a novel framework integrating Bayesian data assimilation with machine learning to enhance forecast accuracy and uncertainty quantification in turbulent systems.

Findings

Significantly improves forecast skill over traditional models.

Effectively quantifies non-Gaussian forecast uncertainty.

Reduces model bias in complex turbulent systems.

Abstract

Ensemble forecast based on physics-informed models is one of the most widely used forecast algorithms for complex turbulent systems. A major difficulty in such a method is the model error that is ubiquitous in practice. Data-driven machine learning (ML) forecasts can reduce the model error but they often suffer from the partial and noisy observations. In this paper, a simple but effective Bayesian machine learning advanced forecast ensemble (BAMCAFE) method is developed, which combines an available imperfect physics-informed model with data assimilation (DA) to facilitate the ML ensemble forecast. In the BAMCAFE framework, a Bayesian ensemble DA is applied to create the training data of the ML model, which reduces the intrinsic error in the imperfect physics-informed model simulations and provides the training data of the unobserved variables. Then a generalized DA is employed for the initialization of the ML ensemble forecast. In addition to forecasting the optimal point-wise value, the BAMCAFE also provides an effective approach of quantifying the forecast uncertainty utilizing a non-Gaussian probability density function that characterizes the intermittency and extreme events. It is shown using a two-layer Lorenz 96 model that the BAMCAFE method can significant improve the forecasting skill compared to the typical reduced-order imperfect models with bare truncation or stochastic parameterization for both the observed and unobserved large-scale variables. It is also shown via a nonlinear conceptual model that the BAMCAFE leads to a comparable non-Gaussian forecast uncertainty as the perfect model while the associated imperfect physics-informed model suffers from large forecast biases.