Forecasting with Guidance: Representation-Level Supervision for Time Series Forecasting

TL;DR

ReGuider is a versatile plug-in method that enhances time series forecasting by aligning encoder embeddings with pretrained models, capturing salient dynamics more effectively than traditional end-to-end training.

Contribution

The paper introduces ReGuider, a novel representation-level supervision technique that leverages pretrained models to improve the expressiveness of temporal representations in forecasting.

Findings

Consistently improves forecasting accuracy across datasets.

Enhances encoder representations with semantic and temporal richness.

Compatible with various architectures and datasets.

Abstract

Nowadays, time series forecasting is predominantly approached through the end-to-end training of deep learning architectures using error-based objectives. While this is effective at minimizing average loss, it encourages the encoder to discard informative yet extreme patterns. This results in smooth predictions and temporal representations that poorly capture salient dynamics. To address this issue, we propose ReGuider, a plug-in method that can be seamlessly integrated into any forecasting architecture. ReGuider leverages pretrained time series foundation models as semantic teachers. During training, the input sequence is processed together by the target forecasting model and the pretrained model. Rather than using the pretrained model's outputs directly, we extract its intermediate embeddings, which are rich in temporal and semantic information, and align them with the target model's encoder embeddings through representation-level supervision. This alignment process enables the encoder to learn more expressive temporal representations, thereby improving the accuracy of downstream forecasting. Extensive experimentation across diverse datasets and architectures demonstrates that our ReGuider consistently improves forecasting performance, confirming its effectiveness and versatility.