BeamVQ: Aligning Space-Time Forecasting Model via Self-training on Physics-aware Metrics

TL;DR

BeamVQ is a novel self-training method that improves the physical realism of space-time forecasting models by filtering and training on physics-aware high-quality samples, boosting both statistical and physical metrics.

Contribution

The paper introduces BeamVQ, a flexible self-training framework that aligns data-driven models with physical laws using physics-aware metrics and beam search sampling.

Findings

Achieved over 32% improvement in statistical skill scores across multiple backbones and datasets.

Significantly enhanced physics-aware metrics, improving physical plausibility of predictions.

Demonstrated versatility across different encoder-decoder architectures.

Abstract

Data-driven deep learning has emerged as the new paradigm to model complex physical space-time systems. These data-driven methods learn patterns by optimizing statistical metrics and tend to overlook the adherence to physical laws, unlike traditional model-driven numerical methods. Thus, they often generate predictions that are not physically realistic. On the other hand, by sampling a large amount of high quality predictions from a data-driven model, some predictions will be more physically plausible than the others and closer to what will happen in the future. Based on this observation, we propose \emph{Beam search by Vector Quantization} (BeamVQ) to enhance the physical alignment of data-driven space-time forecasting models. The key of BeamVQ is to train model on self-generated samples filtered with physics-aware metrics. To be flexibly support different backbone architectures, BeamVQ leverages a code bank to transform any encoder-decoder model to the continuous state space into discrete codes. Afterwards, it iteratively employs beam search to sample high-quality sequences, retains those with the highest physics-aware scores, and trains model on the new dataset. Comprehensive experiments show that BeamVQ not only gave an average statistical skill score boost for more than 32% for ten backbones on five datasets, but also significantly enhances physics-aware metrics.