VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting

TL;DR

VAMoE introduces a dynamic, expert-based framework for incremental weather forecasting that adapts to changing data patterns, reducing computational costs while maintaining high accuracy.

Contribution

The paper proposes a novel Variables Adaptive Mixture of Experts framework that dynamically selects experts for efficient, incremental weather prediction.

Findings

Achieves comparable accuracy to state-of-the-art models.

Uses only 25% of trainable parameters.

Requires 50% less initial training data.

Abstract

This paper presents Variables Adaptive Mixture of Experts (VAMoE), a novel framework for incremental weather forecasting that dynamically adapts to evolving spatiotemporal patterns in real time data. Traditional weather prediction models often struggle with exorbitant computational expenditure and the need to continuously update forecasts as new observations arrive. VAMoE addresses these challenges by leveraging a hybrid architecture of experts, where each expert specializes in capturing distinct subpatterns of atmospheric variables (temperature, humidity, wind speed). Moreover, the proposed method employs a variable adaptive gating mechanism to dynamically select and combine relevant experts based on the input context, enabling efficient knowledge distillation and parameter sharing. This design significantly reduces computational overhead while maintaining high forecast accuracy. Experiments on real world ERA5 dataset demonstrate that VAMoE performs comparable against SoTA models in both short term (1 days) and long term (5 days) forecasting tasks, with only about 25% of trainable parameters and 50% of the initial training data.