Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulence

TL;DR

This paper introduces a convolutional variational autoencoder-based stochastic model for geophysical turbulence that, through transfer learning, achieves improved short-term forecasting and long-term climate stability using limited observational data.

Contribution

It presents a novel transfer learning approach with a stochastic autoencoder model that enhances long-term stability and generalization in data-driven weather models.

Findings

Outperforms baseline deterministic models in short-term skill.

Maintains stability and accurate climatology in long-term climate simulations.

Effective transfer learning from imperfect climate models to observational data.

Abstract

Recent years have seen a surge in interest in building deep learning-based fully data-driven models for weather prediction. Such deep learning models if trained on observations can mitigate certain biases in current state-of-the-art weather models, some of which stem from inaccurate representation of subgrid-scale processes. However, these data-driven models, being over-parameterized, require a lot of training data which may not be available from reanalysis (observational data) products. Moreover, an accurate, noise-free, initial condition to start forecasting with a data-driven weather model is not available in realistic scenarios. Finally, deterministic data-driven forecasting models suffer from issues with long-term stability and unphysical climate drift, which makes these data-driven models unsuitable for computing climate statistics. Given these challenges, previous studies have tried to pre-train deep learning-based weather forecasting models on a large amount of imperfect long-term climate model simulations and then re-train them on available observational data. In this paper, we propose a convolutional variational autoencoder-based stochastic data-driven model that is pre-trained on an imperfect climate model simulation from a 2-layer quasi-geostrophic flow and re-trained, using transfer learning, on a small number of noisy observations from a perfect simulation. This re-trained model then performs stochastic forecasting with a noisy initial condition sampled from the perfect simulation. We show that our ensemble-based stochastic data-driven model outperforms a baseline deterministic encoder-decoder-based convolutional model in terms of short-term skills while remaining stable for long-term climate simulations yielding accurate climatology.