A federated large language model for long-term time series forecasting

TL;DR

FedTime is a federated large language model designed for long-term time series forecasting, addressing privacy, scalability, and communication challenges through clustering, fine-tuning, and semantic preservation techniques.

Contribution

The paper introduces FedTime, a novel federated LLM framework with clustering and alignment strategies for improved long-range time series prediction.

Findings

Significant accuracy improvements over existing methods

Reduced communication overhead in federated settings

Effective preservation of local semantic information

Abstract

Long-term time series forecasting in centralized environments poses unique challenges regarding data privacy, communication overhead, and scalability. To address these challenges, we propose FedTime, a federated large language model (LLM) tailored for long-range time series prediction. Specifically, we introduce a federated pre-trained LLM with fine-tuning and alignment strategies. Prior to the learning process, we employ K-means clustering to partition edge devices or clients into distinct clusters, thereby facilitating more focused model training. We also incorporate channel independence and patching to better preserve local semantic information, ensuring that important contextual details are retained while minimizing the risk of information loss. We demonstrate the effectiveness of our FedTime model through extensive experiments on various real-world forecasting benchmarks, showcasing substantial improvements over recent approaches. In addition, we demonstrate the efficiency of FedTime in streamlining resource usage, resulting in reduced communication overhead.