Multi-Scale Finetuning for Encoder-based Time Series Foundation Models

TL;DR

This paper introduces Multiscale Finetuning (MSFT), a novel framework for encoder-based time series foundation models that explicitly models multiple scales during finetuning, significantly improving forecasting performance.

Contribution

The paper proposes MSFT, a general multiscale finetuning method that enhances encoder-based TSFMs by explicitly incorporating multi-scale modeling, outperforming existing approaches.

Findings

MSFT outperforms naive finetuning methods.

MSFT surpasses state-of-the-art deep learning models.

Experimental validation on three backbones confirms effectiveness.

Abstract

Time series foundation models (TSFMs) demonstrate impressive zero-shot performance for time series forecasting. However, an important yet underexplored challenge is how to effectively finetune TSFMs on specific downstream tasks. While naive finetuning can yield performance gains, we argue that it falls short of fully leveraging TSFMs' capabilities, often resulting in overfitting and suboptimal performance. Given the diverse temporal patterns across sampling scales and the inherent multi-scale forecasting capabilities of TSFMs, we adopt a causal perspective to analyze finetuning process, through which we highlight the critical importance of explicitly modeling multiple scales and reveal the shortcomings of naive approaches. Focusing on encoder-based TSFMs, we propose Multiscale finetuning (MSFT), a simple yet general framework that explicitly integrates multi-scale modeling into the finetuning process. Experimental results on three different backbones (Moirai, Moment and Units) demonstrate that TSFMs finetuned with MSFT not only outperform naive and typical parameter efficient finetuning methods but also surpass state-of-the-art deep learning methods. Codes are available at https://github.com/zqiao11/MSFT.