Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting

TL;DR

Time-FFM introduces a federated foundation model for time series forecasting that leverages pretrained language models, transforming data into text tokens and enabling privacy-preserving, domain-adaptive forecasting with strong few-shot and zero-shot performance.

Contribution

The paper proposes a novel federated foundation model for time series forecasting that transforms data into text tokens and uses dynamic prompts, addressing data scarcity and privacy issues.

Findings

Outperforms state-of-the-art methods in experiments.

Effective in few-shot and zero-shot forecasting scenarios.

Demonstrates robustness across heterogeneous domains.

Abstract

Unlike natural language processing and computer vision, the development of Foundation Models (FMs) for time series forecasting is blocked due to data scarcity. While recent efforts are focused on building such FMs by unlocking the potential of language models (LMs) for time series analysis, dedicated parameters for various downstream forecasting tasks need training, which hinders the common knowledge sharing across domains. Moreover, data owners may hesitate to share the access to local data due to privacy concerns and copyright protection, which makes it impossible to simply construct a FM on cross-domain training instances. To address these issues, we propose Time-FFM, a Federated Foundation Model for Time series forecasting by leveraging pretrained LMs. Specifically, we begin by transforming time series into the modality of text tokens. To bootstrap LMs for time series reasoning, we propose a prompt adaption module to determine domain-customized prompts dynamically instead of artificially. Given the data heterogeneity across domains, we design a personalized federated training strategy by learning global encoders and local prediction heads. Our comprehensive experiments indicate that Time-FFM outperforms state-of-the-arts and promises effective few-shot and zero-shot forecaster.