FeDaL: Federated Dataset Learning for General Time Series Foundation Models

TL;DR

FeDaL introduces a federated learning framework for time series models that effectively handles dataset heterogeneity by learning shared representations and eliminating biases, improving generalization across diverse real-world tasks.

Contribution

The paper proposes FeDaL, a novel federated dataset learning method that explicitly mitigates biases in heterogeneous time series data for better generalization of foundation models.

Findings

FeDaL outperforms 54 baselines across eight real-world tasks.

Federated scaling analysis shows data volume and client number influence performance.

FeDaL effectively reduces domain biases in diverse time series datasets.

Abstract

Dataset-level heterogeneity introduces significant domain biases that fundamentally degrade generalization on general Time Series Foundation Models (TSFMs), yet this challenge remains underexplored. This paper rethinks the from-scratch training of TSFMs using the paradigm of federated learning. We propose a novel Federated Dataset Learning (FeDaL) approach to tackle heterogeneous time series by learning dataset-agnostic temporal representations. Specifically, the distributed architecture of federated learning is a nature solution to decompose heterogeneous TS datasets into shared generalized knowledge and preserved personalized knowledge. Moreover, based on the TSFM architecture, FeDaL explicitly mitigates both local and global biases by adding two complementary mechanisms: Domain Bias Elimination (DBE) and Global Bias Elimination (GBE). FeDaL`s cross-dataset generalization has been extensively evaluated in real-world datasets spanning eight tasks (including various regression and classification), against 54 baselines. We further analyze federated scaling behavior, showing how data volume, client count, and join rate affect model performance under decentralization. Our code is publicly available at https://github.com/shengchaochen82/FeDaL