From Swath to Full-Disc: Advancing Precipitation Retrieval with Multimodal Knowledge Expansion

TL;DR

This paper presents PRE-Net, a novel two-stage pipeline that enhances infrared-based satellite precipitation retrieval to achieve full-disc coverage by transferring knowledge from multimodal data, outperforming existing methods.

Contribution

Introduction of PRE-Net, a two-stage model that transfers multimodal knowledge to improve full-disc infrared precipitation retrieval beyond the scanning swath.

Findings

PRE-Net outperforms PERSIANN-CCS, PDIR, and IMERG.

The two-stage pipeline effectively transfers multimodal knowledge.

Significant improvement in precipitation retrieval accuracy.

Abstract

Accurate near-real-time precipitation retrieval has been enhanced by satellite-based technologies. However, infrared-based algorithms have low accuracy due to weak relations with surface precipitation, whereas passive microwave and radar-based methods are more accurate but limited in range. This challenge motivates the Precipitation Retrieval Expansion (PRE) task, which aims to enable accurate, infrared-based full-disc precipitation retrievals beyond the scanning swath. We introduce Multimodal Knowledge Expansion, a two-stage pipeline with the proposed PRE-Net model. In the Swath-Distilling stage, PRE-Net transfers knowledge from a multimodal data integration model to an infrared-based model within the scanning swath via Coordinated Masking and Wavelet Enhancement (CoMWE). In the Full-Disc Adaptation stage, Self-MaskTune refines predictions across the full disc by balancing multimodal and full-disc infrared knowledge. Experiments on the introduced PRE benchmark demonstrate that PRE-Net significantly advanced precipitation retrieval performance, outperforming leading products like PERSIANN-CCS, PDIR, and IMERG. The code will be available at https://github.com/Zjut-MultimediaPlus/PRE-Net.