Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and Generation

TL;DR

Omni-Weather is a pioneering multimodal foundation model that unifies weather radar understanding and generation, enabling interpretable outputs and improved performance through shared architecture and causal reasoning.

Contribution

The paper introduces Omni-Weather, the first model to combine weather generation and understanding in a single architecture with a Chain-of-Thought dataset for causal reasoning.

Findings

Achieves state-of-the-art in weather generation and understanding

Generative and understanding tasks mutually enhance each other

Demonstrates the feasibility of unifying weather modeling tasks

Abstract

Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.