Federated Prompt Learning for Weather Foundation Models on Devices

TL;DR

This paper introduces FedPoD, a federated prompt learning approach for on-device weather forecasting that enhances model customization and communication efficiency amid data heterogeneity and limited resources.

Contribution

FedPoD proposes adaptive prompt tuning and dynamic graph modeling to improve federated weather models' accuracy and communication efficiency on heterogeneous devices.

Findings

FedPoD outperforms state-of-the-art baselines in real-world datasets.

Adaptive prompt tuning enhances prediction accuracy.

Dynamic graph modeling improves collaboration among similar data distributions.

Abstract

On-device intelligence for weather forecasting uses local deep learning models to analyze weather patterns without centralized cloud computing, holds significance for supporting human activates. Federated Learning is a promising solution for such forecasting by enabling collaborative model training without sharing raw data. However, it faces three main challenges that hinder its reliability: (1) data heterogeneity among devices due to geographic differences; (2) data homogeneity within individual devices and (3) communication overload from sending large model parameters for collaboration. To address these challenges, this paper propose Federated Prompt Learning for Weather Foundation Models on Devices (FedPoD), which enables devices to obtain highly customized models while maintaining communication efficiency. Concretely, our Adaptive Prompt Tuning leverages lightweight prompts guide frozen foundation model to generate more precise predictions, also conducts prompt-based multi-level communication to encourage multi-source knowledge fusion and regulate optimization. Additionally, Dynamic Graph Modeling constructs graphs from prompts, prioritizing collaborative training among devices with similar data distributions to against heterogeneity. Extensive experiments demonstrates FedPoD leads the performance among state-of-the-art baselines across various setting in real-world on-device weather forecasting datasets.