Fine-Tuning a Time Series Foundation Model with Wasserstein Loss

TL;DR

This paper introduces the use of Wasserstein loss for fine-tuning time series foundation models, showing it improves point estimation over traditional cross-entropy loss across multiple datasets.

Contribution

The paper proposes replacing cross-entropy loss with Wasserstein loss in time series models, addressing the mismatch between loss function and task.

Findings

Wasserstein loss significantly improves point estimation accuracy.

Fine-tuning with Wasserstein loss outperforms cross-entropy loss on 22 zero-shot datasets.

The approach enhances the effectiveness of foundation models for time series forecasting.

Abstract

Inspired by recent advancements in large language models (LLMs) for Natural Language Processing (NLP), there has been a surge in research focused on developing foundational models for time series forecasting. One approach involves training LLM architectures on tokenized time series data using cross-entropy loss. Although this method has demonstrated promising results, cross-entropy loss is primarily designed for classification tasks and does not account for the distance between classes. To address this limitation, we propose using the Wasserstein loss for such architectures. To validate our approach, we fine-tuned a foundational time series model on $22$ zero-shot datasets, comparing the performance of cross-entropy loss with that of Wasserstein loss. Our results demonstrate that replacing cross-entropy loss with Wasserstein loss significantly improves point estimation.